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ABSTRACT 

By establishing its Outreach and Technical Assistance Network 
(OTAN) in 1989, California became the first state to spend substantial funds 
on building a technological infrastructure for adult education. OTAN 1 s goals 
are as follows: foster communication among state officials, researchers, and 
local administrators, instructors, and staff in adult education; facilitate 
access to library and other information about adult education policies and 
practices; and provide technical assistance in introducing the use of 
technology to adult educators. OTAN ’ s history may be divided into the 
following periods: (1) building the foundation (1989-1994); (2) building a 

communication and information system (1994-1997); and (3) enhancing the 
system (1997-1999). OTAN ’ s staff has provided training and technical 
assistance to enable its subscribers to feel comfortable using technology and 
to become facile in accessing OTAN's various services and materials. To reach 
a variety of adult education programs throughout California, OTAN 1 s staff has 
provided on-site training in regional resource centers and local programs. 
Work is under way to enhance OTAN ' s roles as a platform for furthering the 
use of technology and as a gateway to adult education resources worldwide. 
Copies of the OTAN user and nonuser surveys and seven graphics depicting use 
of an OTAN Web site are appended. (Contains 12 tables/figures and 10 
references.) (MN) 
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Executive Summary 



Overview of OTAN 

With its investment in the Outreach and Technical Assistance Network (OTAN) beginning in 
1989, the California Department of Education (CDE) was the first state to spend substantial 
funds in building a technological infrastructure for the field of adult education. The developers 
of OTAN envisioned an online system that would: 

• Foster communication among state officials, researchers, and local administrators, 
instructors, and staff in adult education; 

• Facilitate access to library and other information about adult education policies and 



• Provide technical assistance in introducing the use of technology to adult educators; and 

• Serve as a platform for future applications of technology in adult education. 

Originally supported with monies under the Adult Education Act and most recently with funds 
under Title II of the Workforce Investment Act of 1998, OTAN initially was administered by the 
Hacienda La Puente Unified School District and in 1994 became a project of the Sacramento 
County Office of Education. For the past decade, OTAN has functioned as an online 
communication network delivering services in three key areas: electronic communications, 
information and referral services, and instructional technology. As the state of technology has 
advanced, OTAN has extended its system to incorporate new tools and information that have 
enabled the system to move beyond its original intent. 

Abt Associates' Evaluation 

As OTAN began its third contract period in 1997, the CDE and OTAN's director recognized the 
need to examine the quality of online services that were being offered through OTAN and their 
effect on adult education practitioners, policymakers, and programs. In order to document the 
history and development of OTAN as well as answer the critical questions about its 
implementation and use, the Sacramento County Office of Education contracted with Abt 
Associates in Bethesda, MD to conduct an evaluation that would examine: 

• The quantity and perceived quality of the services and materials that OTAN has 



• The ways in which staff in local adult education agencies have utilized OTAN; 

• The effects of OTAN's services on adult education program operations, staff, and policy 
development; and 
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• The ways in which OTAN's experience can inform further development of online 
services and products. 

Abt Associates’ research team chose a multi-method approach for the evaluation that included: 
the review of existing information about OTAN's historic development and use of services, and 
the collection of new data about the perceived quality and use of OTAN's services. This 
gathering of data involved the conduct of three surveys, telephone interviews, and face-to-face 
interviews, as well as the review of OTAN's online services, reports, and other documents related 
to its history and implementation. The data collection period for the evaluation was from May 
1997 through June 1999. 



OTAN's Development 

Phases of Development. OTAN's development has been a phased process that began in 1989. 
Since its inception, OTAN's activities have helped to transform the state-of-practice in adult 
education in the state of California from limited or no access to computers for communication 
and information retrieval to high access. This transformation has occurred in three phases: 

Building the Foundation : December 1989- January 1994. During this period, OTAN's staff 
established an electronic network, provided staff development to local adult education agencies 
to access and use the network, identified and developed information resources for retrieval from 
the network, and conducted outreach to promote the use of OTAN among adult educators; 

Building a Communication and Information System: February 1994-March 1997. As the 
number of subscribers and the ways in which they used OTAN increased, OTAN's staff focused 
on improving the quality of the technical system and the information stored on the system. A 
number of technical upgrades that staff made during this period resulted in a more powerful and 
easier-to-use online communication and information system. Some of these upgrades expanded 
the types of communication that subscribers could undertake, such as the installation of the 
software to host online Email discussions as well as software to provide online multimedia 
content. In addition to the provision of ongoing training and technical assistance, OTAN's staff 
conducted targeted staff development to expand the capacity of key system users, such as adult 
education agency administrators and Regional Resource Center managers. The expansion of the 
library of online materials continued, as did the establishment of linkages with other online 
systems. To enable local agencies to be more systematic in their use of technology, OTAN 
offered technology planning and implementation workshops to adult education providers. 

Enhancing the OTAN System: April 1997 -June 1999. During this period, OTAN's staff made a 
number of changes in the system to respond to state policy shifts as well as subscribers' needs. 
OTAN continued to improve the communication and information system, provided additional 
training and technical assistance, further enhanced the information and reference services, and 
offered outreach and assistance to support technology planning. Shifts in state policy prompted 
OTAN to adjust the electronic communication system in three ways: 
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• To provide free email accounts to agencies receiving state adult education monies; 

• To transition out of the business of providing both Internet access and Email, because of 
the increased number of OTAN subscribers who no longer relied on OTAN for those 
things; and 

• To refine OTAN's Web site to include all components of service. 

Other new developments during this period included the piloting of online staff development in 
conjunction with the state's staff development provider, the development of multimedia 
programs, and the expansion of partnership with entities such as CNN to provide online adult 
learner instruction. In January 1999, OTAN's released the redesigned Web site. In addition to 
the no-cost registration feature, it has an online User Guide to assist inexperienced visitors, a 
Java-based pull-down menu system, a new home page, a redesigned Round Table area, and an 
expanded area called "OTAN For Teachers." 

Summary. Since its inception, OTAN's staff have used state-of-the-art technology to create an 
electronic communications and information infrastructure for serving adult educators. OTAN's 
development has been both methodical and iterative, initially building a foundation for the 
infrastructure, then expanding the types of services and products available in its system, and 
most recently expanding the capacity and linkages of the system. What began as a special 
project to serve adult educators in California developed into a national resource for linking 
people and information electronically. 

Four features have characterized OTAN's evolution: 

• Use of state-of-the-art technology to create and refine the communication system and 
provide services, including ongoing updates to the hardware and software that have 
allowed for expansion and efficiency; 

• Provision of technical assistance and training using a variety of modalities to support all 
levels of adult education program staff, state policymakers, and others in accessing and 
using OTAN; 

• Development of different communication vehicles for fostering information sharing, 
building collegial relationships, and improving adult education practices; and 

• Creation of an information base that includes the development of new materials, 
gathering of existing information, and the linking with other databases to provide 
resources for adult education program management, instruction, training, and policy 
development. 

In undertaking these activities, OTAN's staff have worked to provide quality services in a timely 
manner that have enabled adult educators to access OTAN and have increased the capacity of 
these individuals to use computers. The data that Abt Associates' research team collected 
indicated strong customer satisfaction with the assistance that OTAN has provided, as well as 
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with OTAN’s gradual expansion of materials and assistance to support the system. This 
satisfaction may be attributed both to the methodical approach that OTAN's staff used in 
developing the system, as well as to the staffs attention to the need for ongoing system 
refinement, materials creation, and customer support. This approach has resulted in a system 
with a wealth of resources that can be used in a myriad of ways to improve adult education 
services. 



Technical Assistance and Training 

A critical element in the development of OTAN has been the training and technical assistance 
that staff offered to support the dissemination of the system and encourage its use. The 
assistance that OTAN provided not only enabled local adult education program staff to become 
familiar with the online system and its capabilities, but also played a critical role in building the 
capacity of these staff to use computers. OTAN's staff leveraged their training and technical 
assistance expertise by working with the Regional Resource Centers to deliver assistance to the 
adult education programs in their regions. OTAN's staff also collaborated with these programs 
in hosting training sessions that OTAN delivered. As a result of this systematic approach to 
capacity building, the staff from OTAN and the Regional Resource Centers delivered training to 
approximately 20,000 individuals over the three OTAN grant periods, as is shown in Exhibit 1. 
As more adult educators became familiar with OTAN, the staff continued to provide technical 
assistance so that by the third contract period, OTAN’s staff had responded to more than 25,000 
calls and reference inquires since the beginning of OTAN. 



Exhibit 1: Training and Technical 
Assistance* 
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Through its technical assistance activities, OTAN's staff also have facilitated the work of special 
projects supported by the California Department of Education's Education Support System 
Division. These special projects have linked members of service provider groups, such as the 
library system, and have facilitated the work of curricular development projects. Thus OTAN 
has helped individuals as well as groups of individuals involved in development and assistance 
projects. 



The Utilization of OTAN 

As a result of OTAN's training and technical assistance activities, its subscriber base grew 
substantially over the three grant periods. As shown in Exhibit 2, OTAN had over 2,000 
subscribers by 1999. A variety of factors influenced this growth, including the California 
Department of Education's requirement that each grantee receiving state Section 321 funding 
obtain an OTAN account, as well as the numerous dissemination activities that OTAN's staff 
conducted. In this evaluation, the OTAN subscribers whom we surveyed and interviewed 
expressed high satisfaction with the training and technical assistance they received as well as 
with the resources they were able to access. The key barriers to use were the quality of the local 
technology and subscribers' time to use the system. 




OTAN's primary goal was to establish a communication system that would link all stakeholders 
in the state's adult education system. Our evaluation data substantiate OTAN's achievement of 
this goal, particularly with regard to subscribers' exchange of information concerning 
administrative and compliance issues. The California Department of Education has facilitated 
this exchange by increasingly using OTAN to communicate with its grantees and to process 
materials, such as the approval of courses and the applications for state funding. OTAN has 
become the platform for information on key issues affecting the operation of state-funded adult 
education agencies. 
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As OTAN expanded its services, its role in promoting program improvement developed. In 
addition to posting course outlines and lesson plans created by California adult educators as well 
as research and library documents, OTAN became the gateway to instructional information 
available on the Internet. With the redesign of OTAN's Web site in 1999, general resources as 
well as those targeted to teachers were better organized and more accessible to subscribers. By 
1999, OTAN's collection contained approximately 20,000 resource documents, as is illustrated in 
Exhibit 3. To promote subscribers' use of these resources, OTAN's staff have highlighted on the 
Web site key resources and activities both within state and nationally that can assist program 
staff in developing and delivering quality instruction. 



Exhibit 3: Growth of OTAN Resources 
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While OTAN has great potential as a vehicle to support program improvement, this potential is 
still developing. In our interviews with program administrators, we found that many were 
involved in improvement activities and had accessed OTAN's resources to assist them. These 
activities included revising curriculum, implementing assessments, investigating new 
instructional strategies, and making changes to program policies and procedures. The 
administrators revised curriculum by accessing OTAN’s resources such as Course Outlines, 
Lesson Plans, or the Round Table. Rather than creating new products, the information available 
through OTAN enabled administrators to build upon the work of others in developing their 
materials. They also accessed the Exchange or the Round Table areas to pose specific questions 
to other subscribers about their incorporation of different instructional materials and methods. 

OTAN’s capacity as a communication vehicle also facilitated administrators’ program 
improvement activities. A common issue of interest was how technology can be used to 
facilitate student learning. OTAN’s Exchange and Round Table areas were resources that 
administrators accessed to communicate with their colleagues about strategies for implementing 
technological tools. The Adult Ed and Technology Vendor Databases also provided helpful 
information about products and services related to the use of technology. 
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Staff development is a critical component of program improvement, and OTAN’s 
announcements of workshops, conferences, and teleconferences provided timely information to 
administrators and their staff about opportunities to learn new practices. OTAN’s materials also 
served as a source of information that staff could use in creating training events on topics such as 
strategies for using the Internet and developing lesson plans. 

Another activity related to improvement in which OTAN aided administrators was program 
management. As adult education programs sought to expand their service capacity and 
incorporate the use of technology, staff needed access to the latest information about funding 
sources, legislative initiatives, and technology product development. Through OTAN, 
administrators were able to keep abreast of pending and new legislation, as well as a variety of 
public and private funding sources. OTAN was a virtual one-stop resource that was efficient to 
use and easy to access. 

While administrators accessed OTAN in undertaking various improvement activities, teachers' 
use of OTAN for improvement purposes was less prominent. Many teachers are self-motivated 
and have incorporated OTAN's resources as a matter of practice, while others may need external 
direction to prompt their use. A strategy used by some state adult education programs to 
promote teachers’ improvement activities has been to sponsor a statewide improvement initiative 
in which the transformation of teachers’ practices has been the focal point. OTAN could play a 
vital role in such an initiative, both in providing information about new practices and in enabling 
teachers to share their experiences with each other. 

As a communication and information platform, OTAN has provided both resources and 
mechanisms for exchange among adult educators. The staff also have linked California's adult 
educators to resources nationally to encourage the implementation of the best possible adult 
education services in the state. While OTAN can play a critical role in facilitating quality 
practice among adult educators, staff from the State Department of Education and local adult 
schools also need to be instrumental in fostering program improvement and OTAN's use in this 
process. 



Future Directions 

The California Department of Education broke new ground in funding OTAN, and it has been a 
unique service in the field of adult education. OTAN also served as a model for the National 
Institute of Literacy's development of LINCS-the Literacy Information aNd Communication 
System. At this stage of adult education policy and practice in California and nationally, OTAN 
has potential in serving two key roles: as a platform to furthering the use of technology and as a 
gateway to adult education resources worldwide. 

Platform for Furthering the Use of Technology. OTAN has functioned as an information 
dissemination and communication base in addressing the needs of the California Department of 
Education and state-funded adult education programs. As these needs expand and as new 
developments in technology come online, OTAN has a key role in the following: 
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• Expanding its services in facilitating the California Department of Education's 
accountability and grants administration activities. As state-funded adult education 
programs submit their accountability data online, OTAN could promote the local use of 
these data for program management and improvement. By expanding its online training 
activities, OTAN could provide simulation exercises in using data that local program 
staff could access. OTAN also could stimulate subscribers' exchange of information 
concerning the ways in which student data and other program information could be 
analyzed and used for program purposes. As the California Department of Education 
revises and enhances its grants management processes, OTAN has a key role in 
disseminating this information and serving as a mechanism for data collection; and 

• Disseminating new technology products funded by the U.S. Department of Education 
(ED). OTAN has been a collaborating partner in the development of the U.S. 

Department of Education-supported Cyberstep project. This project is one of three 
cutting-edge developments in online instruction, staff development, and assessment that 
ED has funded and which will be ready for national dissemination this year. As these 
products become available, there is a need to orient the field of adult education to the 
ways in which these products could be incorporated into ongoing program services. 
OTAN is well suited to serve as a platform for orienting adult education practitioners to 
the use of these products, as well as to linking practitioners to other resources that could 
support their implementation. 

Gateway to Adult Education Resources. OTAN has a unique role to play in directing 
subscribers to the myriad of resources that can be accessed through its system. One critical use of 
these resources is in facilitating the delivery of quality adult education services. Two 
recommendations for OTAN's role in program improvement are: 

• Revise the format for the submission of lesson plans to include sections regarding the 
teacher's theory or guiding principles and related instructional materials. The inclusion of 
information about the theory or principles will help others to understand the developer’s 
instructional perspective in creating the lesson and how it might be applied. Information 
about related materials will guide others in selecting appropriate resources and 
applications that match the intent of the lesson; and 

• Use a problem-based method to stimulate adult education practitioners' improvement of 
practice. Abt Associates' work with the statewide program improvement initiative in 
Pennsylvania— Project EQuAL— demonstrated that adult education teachers and program 
administrators are motivated to address the operational problems that impede their 
delivery of quality program services. Using the problem identification process as a 
stimulant, staff are able to analyze information and use resources to try new practices. 
OTAN could adapt the problem-based technique by posting common problems that adult 
education practitioners face in managing and teaching as a way of engaging practitioners 
in program improvement. OTAN subscribers could be directed to the appropriate online 
resources and documents that relate to the problems that are posted. While many 
documents are not written to provide direct solutions to program issues, the information 
in the documents could stimulate ideas and new directions. Through OTAN's Exchange 
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and Chats functions, subscribers could share their solutions and suggest new practices. 
This process could serve as a starting point for adult education programs to use in 
addressing their operational issues. 

These recommendations are illustrative of the potential that OTAN has for enhancing its 
activities in facilitating communication and program improvement. OTAN has a critical 
function in disseminating information from the California Department of Education and in 
facilitating the state's grants management process. Its role in obtaining documents and library 
resources also is expanding from a retrieval and storage function to a linking function. As a 
platform for technology, OTAN can direct subscribers to effective use and application of existing 
and new technological tools. As a gateway to information, it can guide subscribers in making 
informed choices regarding the variety of resources that are available to adult educators 
worldwide. 
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I. Introduction 



Overview of the Evaluation 

Impetus for the Evaluation. With its investment in the Outreach and Technical Assistance 
Network (OTAN) beginning in 1989, the California Department of Education (CDE)’ was the 
first state to spend substantial funds on building a technological infrastructure for the field of 
adult education. The developers of OTAN envisioned a technology system that would foster 
communication among adult education state staff, local adult education administrators, 
instructors, and staff; facilitate access to library and other information about adult education 
policies and practices; provide technical assistance in introducing the use of technology to 
adult educators; and serve as a platform for future applications of technology in adult 
education. Given the state-of-the-art of technology and its state-of-use in adult education in 
1989, OTAN’s developers chose an iterative approach in designing OTAN that would mirror 
the ever-changing developments in technology while moving the field of adult education 
toward a state of readiness for using this evolving tool. 

As OTAN began its third contract period in 1997, the California Department of Education 
and OTAN’s director recognized the need to examine the quality of the online services that 
were being offered through OTAN and their effects on adult education practitioners, 
policymakers, and programs. While OTAN had documented the amount of access to its 
services, it had collected only limited data about the implementation of these services and 
their perceived value and results. After almost seven years of implementation, the CDE and 
OTAN staff thought that much could be learned from OTAN’s experience which could 
inform the delivery of online services as well as the design of future products and 
applications. Thus CDE and OTAN determined that a third-party evaluation should be 
procured to document the history and development of OTAN and to answer the critical 
questions about its implementation and use. 

Purposes of the Evaluation. In May 1997, the Sacramento County Office of Education 
(SCOE) awarded a contract to Abt Associates Inc. to conduct an evaluation that had the 
following purposes: 



• To identify the various users’ perceptions of the quality of training and technical 
assistance they had received, as well as the quality and use of the resources and 
materials they had accessed; 

• To examine the implementation of online services that had occurred and the 
perceived impact of the use of these services on staff capacity and program 
operations; and 
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• To identify the state policies and guidance that may be needed as OTAN’s 

services and products expand, as well as lessons that the Federal adult education 
offices and other states might consider in funding and expanding the use of 
technology in the field. 



In specifying these purposes for an evaluation, OTAN’s staff wanted to determine the extent 
to which their goal of developing an online communication system whereby adult educators 
in the state of California had the skill and capacity to access and share information had been 
accomplished. They also wanted to understand the ways in which OTAN’s subscribers had 
utilized its services, and how these services might be improved. Finally, as OTAN’s staff 
moved forward in developing new applications, they wanted to know how OTAN could best 
serve as a platform for their ongoing development and use of technology in adult education. 



Evaluation Approach and Methods 

Framework for Data Collection. In order to address the three purposes of the evaluation, 
Abt Associates’ research team designed a framework for data collection based on the three 
components of OTAN’s services: electronic communications, information and referral 
services, and instructional technology. The questions that guided our inquiry in each 
component are the following: 



• Electronic Communications: What types of training and technical assistance 
have been provided, to whom, and by whom? What is the perceived quality of 
these services? To what extent have the electronic communications services been 
expanded and utilized? What is the perceived impact of these services? What 
factors have facilitated or impeded the use of these services, and what might be 
done to increase their use? 

• Information and Referral Services: What types of services have been offered 
and to whom? To what extent have these services been expanded? What has 
been the perceived utility of these services? 

• Instructional Technology: What types of materials have been developed? What 
is the perceived use of the training and technical assistance that has been 
provided? What factors have facilitated or impeded the use of materials? In what 
ways have the various online services and materials been integrated into an adult 
education program? 



As the main types of services that OTAN provides, these components encompass a variety of 
activities related to materials and system development, training and technical assistance, and 
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refinement and revision. While the framework for the evaluation had to reflect the types of 
services that OTAN carried out, it also had to account for the process activities that OTAN 
performs in developing and delivering its services. Thus we designed an approach to 
studying OTAN that took into consideration both of these aspects of its history and 
implementation. 

Methodology. Abt Associates’ research team chose a multi-method approach to address the 
three purposes of the evaluation that included: the review of existing information about 
OTAN’s historic development and use of its services; and the collection of new data about 
the perceived quality and use of OTAN’s services. This gathering of data involved the 
conduct of three surveys, telephone interviews, and face-to-face interviews, as well as the 
review of OTAN’s online services, reports, and other documents related to its history and 
implementation. We collected data from multiple stakeholders, such as local program 
providers and administrators, state officials, Regional Resource Center staff, OTAN staff, and 
others who used services, developed materials, and provided technical assistance and 
training. We also gathered information using an online method to test the capacity of the 
technology in this regard. 

This approach was designed to produce cross-sectional as well as longitudinal information. 
Our intent was to develop a comprehensive view of OTAN’s past and present so that we 
could formulate recommendations for guiding its future. The data collection activities that we 
undertook were the following: 

Surveys: 

• Conduct of surveys (1997-1998) of 910 subscribers, of which there were 353 
respondents (a response rate of 39 percent) — 252 of whom completed the User 
Survey and 101 completed the Non-user survey (respondents for the Non-user 
survey were determined based on their self-reported minimal or non-use of 
OTAN); 

• Conduct of a Follow-Up Survey (1999) of 226 participants in the 1997-1998 User 
Survey, of which there were 176 respondents (a response rate of 78 percent); 



Interviews: 

• Conduct of telephone interviews (Summer 1998) with 29 OTAN subscribers (18 
administrators, four instructors, four special users, and three state officials); 

• Conduct of face-to-face interviews (1997 and 1998) with state officials, OTAN 
staff, and Section 353 project directors; 
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• Conduct of focus group (1998) with Regional Resource representatives and key 
OTAN users; 

Online Data Collection: 

• Online posting of stimulation questions (1999); 

• Online follow-up with OTAN trainees (1999); 

Document and Product Review: 

• Review of client and satisfaction surveys conducted by OTAN (1991-1999); 

• Review of OTAN Evaluation Reports (1989-1999); 

• Review of selected OTAN Work Group, News Postings, Course Outlines and 
Lesson Plans; and 

• Review of Web and Other Usage statistics (1991-1999). 

Copies of the OTAN User Survey, Non-user Survey, the 1999 Survey are presented in 
Appendix A. The array of data that we collected provided a thorough understanding of 
OTAN’s approach and activities which we used in addressing the questions posed for the 
evaluation. These data also guided the recommendations that we have made for OTAN’s 
future directions. 



Organization of the Report 

Section II presents the history and development of OTAN according to its three contract 
periods. Discussed in Section III is the use of OTAN as a communications network and as a 
resource for information. The training and technical assistance provided by OTAN is 
discussed in Section IV, followed by the conclusions and future directions that are presented 
in Section V. 
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This section summarizes the major milestones and activities in the development of the 
Outreach and Technical Assistance Network from its inception in 1989 to 1999. These 
activities are discussed in terms of the three time periods that reflect OTAN’s contracts: 1) 
December 1989 to January 1994 (four years), 2) February 1994 to March 1997 (three years), 
and 3) April 1997 to June 1999 (two years). 



Overview of OTAN 

OTAN was created in 1989 as a critical component of the implementation of the California 
State Plan for Adult Basic Education, which was required under the Adult Education Act 
(AEA), P.L. 100-297 as amended by the National Literacy Act, P.L. 102-73. This State Plan 
was designed to address the long-term needs and goals identified by California’s Adult 
Education Strategic Plan, which was prepared by the California Department of Education to 
guide the development and implementation of services for the state’s adult education system. 
One component of the State Plan was the provision of training, informational, and technical 
support for agencies receiving Local Assistance Grants under Section 321 of the AEA. In 
response to this requirement, the CDE funded OTAN using AEA Section 353 monies as one 
of the projects that could provide these services and resources to literacy and adult education 
providers in California. Originally administered by the Hacienda La Puente Unified School 
District, OTAN became a project of the Sacramento County Office of Education in 1994. 



Building the Foundation (December 1989 - January 1994) 

OTAN’s first four years were dedicated to building a foundation for an electronic 
communication and information infrastructure in California. Its activities were centered 
around four components: 1) establishing an electronic network, 2) providing staff 
development, 3) providing access to information, and 4) conducting outreach. Exhibit II- 1 
presents a summary of the key activities that OTAN carried out during each of its three 
contract periods. These activities are described below. 

Electronic Network. In 1990, OTAN established an Online Communication System that 
electronically connected adult educators throughout California, allowing them to 
communicate with one another and access a wealth of information and online resources. 

Prior to OTAN’s establishment, this type of electronic system did not exist. With the 
inception of OTAN, the California Department of Education encouraged adult education 
agencies receiving state funding (referred to as Section 321 or Section 321/326 funding) to 
purchase at least one OTAN account, since it was CDE’s intent to use OTAN as the primary 
means of communication with its 32 1 agencies. Agencies also were prompted to use their 
321 funding for investments that would enable them to access OTAN, such as computers and 
telephone lines. 
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Summary of Major Milestones and Activities in OTAN’s Development 
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To facilitate the use of OTAN, agencies were allowed to spend Section 321 funds to pay for 
the one-time $99 OTAN subscriber fee and the monthly access fees charged by Connect, Inc., 
the company that initially hosted OTAN Online. Access fees consisted of a $ 12/month charge 
for an hour of online time, as well as $ 14/hour for each additional hour, prorated to the minute. 

Subscribers accessed OTAN Online by dialing into Connect’s servers using a computer 
modem connected to a telephone line. The software that subscribers used to connect to OTAN 
Online was purchased from Connect and rebundled by OTAN, which also gave subscribers a 
customized user manual and an installation video. 

Initially, the most widely used feature of OTAN Online was the Email system, which allowed 
subscribers to communicate quickly with adult education colleagues and CDE staff, who also 
had OTAN accounts (OTAN, 1994). The system’s group addressing feature allowed OTAN 
and CDE staff to send messages to various combinations of users: all users, all California 
users, or all users receiving Section 321 funding. 

To further encourage communication and collaboration, OTAN established the Round Table 
area, which consisted of workgroups dedicated to particular topics or projects. Each 
workgroup had a message area where subscribers could share messages, post questions, and 
exchange files. Some workgroups were open to all OTAN subscribers, while others were 
accessible only to certain individuals. Early workgroups included the ESL Teacher Institute’s 
Mentor Teacher Training Program, the Comprehensive Adult Student Assessment System 
(CASAS), and the Adult Leadership Training Program. 

While Email and the Round Table workgroups were the major components of OTAN’s 
communication system, the crux of its information storage and retrieval system was the OTAN 
Forum, established in 1990. The Forum consisted of text files in a searchable database that 
was housed on a Connect Inc. server. To make these electronic resources available on the 
Internet, the Forum was duplicated on a Gopher 1 server at Hacienda La Puente School District. 
The Gopher version of the Forum was run parallel to the Connect version until the first OTAN 
Web site was established in 1994. Later, the Gopher version was discontinued and moved to 
an OTAN Web server when the Web became more prominent. The Forum is described in 
greater detail in the “Accessing Information” section. 

During OTAN’s first years, Email and the Internet were relatively new phenomena, used 
mainly on college campuses and in government agencies. The system that OTAN created was, 
for most subscribers, their first exposure to electronic communication and information 
retrieval. Therefore, the introduction of this technology required that OTAN’s staff conduct a 



A Gopher system is a simple client-server protocol that connects users to text files, downloadable files, file searches, or 
to other menus. The Gopher application was very easy to use and thus very popular in the early 1990s, but has since 
been eclipsed by the World Wide Web as the leading interface for navigating the Internet. 
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considerable amount of training and technical assistance. OTAN anticipated and responded to 
this need by creating paper User Guides (available for Macintosh and DOSAVindows 
platforms), conducting training workshops (see “Staff Development” below), and providing 
free and unlimited technical assistance over the phone. By 1994, technical support staff were 
responding to more than 150 phone calls per quarter (OTAN, 1994). 

By its fourth year, OTAN had 435 subscribers. Of these, 239 were from California’s Section 
321 -funded agencies, 82 were from other California agencies, and 57 were guest users from 
other states. 

Staff Development. When OTAN established its electronic network in 1990, computers were 
considerably less prevalent in adult education settings than they are today. Although data are 
not available describing levels of technology use during the early 1990s, a 1993 report by the 
U.S. Office of Technology Assessment estimated that no more than 15 percent of programs 
serving adult learners were regularly using computers for instruction (U.S. Congress, 1993). 
Recent surveys (OTAN, 1998a; Hopey, 1996) have indicated the increase in use of computers 
for both instructional and administrative purposes in adult education. However, due to the 
relative novelty of Email and the Internet in the early 1990s, very few adult educators in 
California regularly used Email or accessed the Internet prior to the establishment of OTAN. 
Therefore, training and technical assistance were key priorities for OTAN. Not only did 
subscribers need technical knowledge to access and use OTAN Online, they also required a 
level of comfort and confidence with computer use in general in order to maximize the 
utilization of OTAN’s online resources. 

OTAN worked with 1 1 Regional Resource Centers 2 to provide training and technical 
assistance to practitioners across the state. Each Center had responsibility for offering services 
that reflected the needs of the region it served. Resource Centers conducted their own needs 
assessments to determine the types of assistance that local agencies needed. During OTAN’s 
first four years in operation, a total of 923 staff development activities were organized for 
nearly 20,000 participants, and Resource Center staff responded to more than 13,000 technical 
assistance queries over the phone or online. The types of training offered varied by region, but 
overall the most popular sessions were related to technology— using productivity software 
(word processing, spreadsheets, etc.) and using OTAN Online (OTAN, 1994). 

In addition to providing training and technical assistance, OTAN worked with the Resource 
Centers in hosting and promoting professional development activities in collaboration with 
other organizations. For example, the Centers hosted CASAS training meetings. Adult 
Literacy Instructor Training (ALIT) Institute sessions, training sessions for Job Training 



Eventually, the number of Resource Centers was reduced from 1 1 to seven due to a reduction in state resources. The 
number later increased to eight. The Resource Centers operated through June 30, 2000. 
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Partnership Act (JTPA) providers, and regional Youth, Adult, and Alternative Education 
Services (YAAES) division meetings. 

Training also was made available though the two Educational Technology Centers (ETC) 3 that 
OTAN created. The ETCs were established primarily to assist adult educators in making 
informed choices about investments in instructional technology. Because they contained 
computer equipment and software, the ETCs became logical places for hands-on training. 
During 1990-94, the ETCs provided more than 3,500 hours of technology-related staff 
development for adult educators. 



Accessing Information. OTAN’s third major charge was to provide informational resources 
for adult educators and other interested stakeholders. To accomplish this, OTAN developed 
principles for information sharing that guided the formation of a national network of 
information sources. These principles were: 



• Instant access to information— the latest research, program innovations, curriculum, 
instructional materials, and resources must be instantly available to California adult 
educators to provide them with the tools to meet the current challenges; 

• Statewide/national network— must be maintained for cost effective, efficient sharing of 
information; 

• Utilization of the latest technological advances— the information accessing effort must be 
on the cutting edge in utilizing the latest technological advances; and 

• Preservation of the reports and products of past efforts— the reports and products of past 
efforts must be preserved so today’s adult education practitioners, researchers, and 
policymakers can build on the successes of their predecessors and avoid their mistakes. 



Information Sharing and Acquisition. OTAN staff developed and maintained relationships 
with several organizations and agencies across the state and nation for sharing adult education 
resources. OTAN’s national network included: 



• U.S. Department of Education— Office of Vocational and Adult Education and Office of 
Educational Research and Improvement; 

• ERIC Clearinghouses; 



3 Originally, the two ETCs were located in northern and southern California. They were later consolidated 
into one Center, located at OTAN’s central office in Sacramento. 
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• Professional associations, such as Teachers of English to Speakers of Other Languages 
(TESOL), American Association of Adult and Continuing Education (AAACE), 
Association of California School Administrators (ACSA), and others; 

• Centers and clearinghouses, such as the National Center on Adult Literacy (NCAL), 
National Center for Family Literacy (NCFL), and others; 

• Other federally funded adult education projects in California, including CASAS, Adult 
Education Institute (AEI), Adult Literacy Instructor Training (ALIT), and others; and 

• Private programs and agencies, such as the Business Council for Effective Literacy and . 
California Literacy. 



Beyond this network, OTAN used several methods to acquire relevant information for its 
subscribers, including reviewing the Federal Register for grant information, monitoring 
newsletters and reports, attending regional, state, and national conferences, and subcontracting 
for specialized information services such as California legislative updates. OTAN’s staff also 
shared information with other online communication systems, including CAVIX (California 
Vocational Information Exchange), ED>NET (Economic Development Network), NCBE CIS 
(National Clearinghouse for Bilingual Education Computer Information Service), and FEDIX 
(Federal Information Exchange). 

Most of the information that OTAN’s staff obtained was posted to the OTAN Forum, the 
online repository of electronic documents and databases. The Forum consisted of several sub- 
areas, including the following: 

• Reference Areas 

- Lesson Plans 

- Course Outlines 

- Current Documents 

- Curricular Resources 

- Software for Macintosh and DOS platforms (public domain and demo software) 

- Library and Clearinghouse Listings (including VESL/Workplace Clearinghouse) 

- Codes and Regulations; and 

• Topical Areas 

- Master Calendar (events in adult education by date, region, topic) 

- CDE Info (notices and directory information from CDE) 

- OTAN Resource Centers (training activities, lists of library holdings) 

- Legislative Information (notices regarding adult education-related legislation) 

- Educational Grants (federal and private funding opportunities) 

- Who’s Who (directory of OTAN subscribers, California 321 agencies and staff, etc.) 
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These Forum areas were identified after OTAN’s staff conducted a needs assessment of 
potential users, and were validated by several subsequent user surveys. By 1994, the Forum 
contained nearly 8,000 separate documents that subscribers could access in two ways. First, 
they could search summaries of the documents by entering keywords that described the topics 
they were trying to access. Second, they could simply browse through each of the Forum’s 
sub- areas. 

Electronic Storage and Retrieval of Information. After OTAN’s staff identified information 
appropriate for the Forum, they had to prepare the material for electronic storage. This 
preparation involved several steps. First, information was evaluated for its permanency. 
Information that was ephemeral and time-sensitive, such as grant alerts and meeting notices, 
went directly to document processing, while permanent and non time-sensitive information 
like curricula and research reports were cataloged into a MARC 4 compatible database. During 
document processing, electronic documents were converted into text only (ASCII) format and 
print documents were scanned and later formatted with word processing software. After 
documents were prepared, they were uploaded into the appropriate Forum area. Occasionally, 
Email messages were sent to subscribers to alert them to significant new additions. During its 
first three-and-a-half years, users opened online Forum documents more than 73,000 times 
(OTAN, 1994). 

Other Resource Collections: Hardcopy and Technology. OTAN developed a number of 
collections in addition to its key reference and topical areas. For example, each Resource 
Center had a Library containing training modules, instructional materials, videos, and other 
documents. During the Centers’ first three years in operation, library materials were accessed 
more than 12,000 times by adult educators. The OTAN central office supported the Resource 
Centers by cataloging deposited items and providing technical assistance as needed. 

OTAN also developed the Adult Education Archives— an historical collection of handbooks, 
reports, research findings, curricula, and staff development and assessment materials. OTAN’s 
staff collected and cataloged these materials, posting abstracts in the Forum for subscribers to 
browse through or search. By early 1994, the collection had grown to more than 500 items, 
which included California Department of Education handbooks and research reports, as well as 
products from state-funded curricula, staff development, and assessment projects. 

The libraries at OTAN’s Educational Technology Centers featured a variety of instructional 
technologies, including hardware, software, computer manuals, and computer peripherals. The 
ETCs allowed adult school personnel to examine software and hardware before deciding which 
to purchase. OTAN’s staff offered several hands-on software reviews and were available to 



4 MARC = Machine-Readable Cataloging format. MARC is the international standard for entering bibliographic records 
in a computer database. Using the MARC format assures that a library database can be integrated with any standard 
library service or search system. 
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answer specific questions about the products. By early 1994, OTAN’s had cataloged more 
than 400 ETC Library items. 

The increasing number of limited-English-proficient (LEP) adults offered new challenges for 
California’s vocational educational programs and workplaces in the 1990s. To address this 
need, OTAN worked with the San Diego Community College District to establish a California 
VESLAVorkplace Clearinghouse. The purpose of the Clearinghouse was to acquire, publicize, 
and distribute vocational educational materials for LEP adults. In its initial year-and-one-half 
in operation, the Clearinghouse maintained 151 items and filled 333 orders. 

Outreach. For its Outreach component, OTAN attempted to increase adult educators’ access 
to services and resources in areas of California that were underserved due to a lack of funds 
and/or geographic isolation. For example, OTAN provided technical assistance to help create 
and support the Tehama-Glenn Adult Literacy Consortium, also known as Project Adult 
Learning (PAL). The Consortium’s primary purpose was to extend educational opportunities 
to geographical areas that were previously unserved. It represented a cross-agency 
collaboration that offered expanded services to key target groups (e.g., displaced workers, 
AFDC recipients) in a variety of modalities (e.g., home-based instruction, computer-assisted 
instruction with laptop and networked computers). 

OTAN also entered into a partnership with the Educational Telecommunications Network in 
Los Angeles County to create the Adult Learning Channel, a mechanism to provide staff 
development activities to adult educators throughout California via satellite. In addition to 
helping produce the Channel’s programming, OTAN coordinated activities at downlink sites 
and promoted the events with mailings and online postings. An estimated 300 educators 
participated in the training. 



Building a Communication and Information System (February 1994 - March 
1997) 

OTAN’s period of operation from 1994 to 1997 was characterized by sharp increases in the 
number of subscribers and their use of OTAN. The OTAN system also continued to evolve. 
Several technical upgrades and improvements resulted in a more powerful and easier-to-use 
online communication and information system. Through its training and technical assistance, 
OTAN demonstrated how the integration of technology can strengthen communication and 
collaboration. During this period, OTAN’s activities were focused on three areas: 1) electronic 
communication, 2) information and reference services, and 3) technology planning and 
implementation. 

Electronic Communication. In 1995, OTAN implemented a major change by transitioning 
from an outdated communication system to a more powerful and manageable Internet-based 
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system. Previously, subscribers accessed OTAN through a proprietary system (Connect, Inc.) 
that was relatively costly given its limited features and slow speed. It became clear that an 
Internet-based system, located at OTAN’s offices and maintained by OTAN staff, would be 
necessary to implement all the communications and information features that subscribers 
needed. 

As a result of the changes that OTAN made, subscribers could access the new Internet-based 
OTAN Online through a customized version of FirstClass® software from SoftArc, Inc. This 
version supported OTAN’s primary features— Email, Round Table work groups, information 
retrieval systems— in addition to several other features that were not possible on the Connect, 
Inc. system. These included access to electronic list servers, news groups, chat groups, and 
downloadable software. Versions for both Macintosh and IBM-compatible computers were 
available. 

In addition to implementation of the Internet-based OTAN Online, OTAN made a number of 
other improvements during this period, including: 

I 

• Installing a high speed Unix World Wide Web (WWW) server and related software; 

• Developing an OTAN Web site (www.otan.dni.us), which provided another avenue for 
subscribers to communicate and find resources; 

• Implementing a 100 Mbps internal local area network to facilitate quick access to both 
OTAN WWW and OTAN Online; 

• Establishing a Wide Area Information Server (WAIS) 5 , used to perform online searches of 
OTAN’s electronic databases and documents; 

• Installing Email list server software to host online Email discussions; 

• Installing RealAudio and RealVideo software to provide online multimedia content; 

• Implementing an online Web-creation system (WebGenerator) to allow novices to create 
their own Web pages; and 

• Developing the Online Adult Education Course Approval (A22) system to facilitate the 
CDE adult education course approval process. 



The improvements gained by moving to the Internet based First Class system, however, were 
counterbalanced by the inconvenience and confusion of creating a dual system— OTAN Online- 



WAIS is a network information retrieval service that searches for key words or phrases in specially indexed files. 
Unlike Gopher, which searches files by their titles, WAIS engines search the full text of files and return a list of 
documents that contain the key word being sought. Most search engines on the World Wide Web use the WAIS 
method of search and retrieval. 
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-continuing the tradition of the proprietary system for Email and online collaboration, and 
OTAN WWW-the Web site for the electronic files and databases in the OTAN Forum. While 
the text files of the Forum were maintained both on First Class and the Web site for a short 
transition period, the Forum soon moved exclusively to the Web in order to implement HTML 6 
and relational database delivery of information. The software capability did not address 
OTAN's needs for several years. Until OTAN moved to one integrated system exclusively on 
the Web in 1999, subscribers had to use two software packages— both the Email software and 
an Internet browser— to access all the features of the system. 

Growth in Subscriber Base and Usage. The 1994 to 1997 period also was one of tremendous 
growth for OTAN. The number of OTAN subscribers nearly tripled, from 435 in early 1994 to 
nearly 1,200 in early 1997. Forum documents were accessed nearly 140,000 times (compared 
with 70,000 times during the previous four years). Use of OTAN WWW— measured in “hits” 
that are actions on the Web site such as opening a page or downloading a document— also 
increased dramatically. From late 1995 to early 1997, the number of hits per month had risen 
from 9,600 to 229,000. 

Three factors can explain this growth. First, computer hardware and software gradually 
became less expensive as the speed and ease of use increased, making is possible for more 
adult education programs to invest in the equipment necessary to access OTAN Online and 
WWW. 7 Second, OTAN’s attention to both customer service and continuous improvement of 
its online systems resulted in high subscriber satisfaction, as evidenced in the results of user 
surveys that OTAN conducted during this time period (OTAN, 1997a). Third, the California 
Department of Education played an important role in encouraging programs to subscribe to and 
regularly use OTAN. As CDE realized that adult educators were accessing OTAN, it began 
including a space on its funding application for applicants to write their OTAN Online ID, 
implicitly encouraging applicants to have an OTAN account. 

Technical Assistance and Training. With its changes and new features, OTAN had to update 
and disseminate its print and non-print training and manuals. These materials included Email, 
file uploading, address books, personal mail folders, OTAN Forum, and using the Web. 

OTAN also created a regular newsletter, which contained tips and answers to frequently asked 
questions. 



HTML = HyperText Markup Language, the programming language used on the World Wide Web. 

A 1995 survey conducted by OTAN measured the level of technology use in adult education programs in California. It 
found that 98 percent of programs were using computers for administration, 83 percent were using them for instruction, 
and 78 percent were using them for electronic communication. 
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OTAN continued to provide a variety of staff development and technical assistance. During 
this three-year period, 2,299 people completed in OTAN workshops and 466 participated in 
classroom training sessions (OTAN, 1997a). 

OTAN’s staff also responded to nearly 4,000 technical support calls (OTAN, 1997a). 

Although most calls were related to installing and using OTAN Online or navigating OTAN 
WWW, several concerned more general problems, such as accessing the Internet, solving a 
modem problem, or deciding which hardware or software to purchase. For some subscribers, 
particularly those in small programs without technical staff, OTAN had become the primary 
source for assistance and advice about computer-related issues. The written Client Satisfaction 
Surveys conducted by OTAN during this period revealed a very high level of satisfaction with 
OTAN’s technical assistance. 

OTAN was proactive in offering technical assistance. For example, after monitoring the use of 
OTAN Online, staff would conduct telephone interviews with subscribers who appeared to 
have problems connecting to the OTAN Online. To provide another avenue of assistance, 
OTAN created a database-driven online Help Desk on OTAN WWW, offering solutions to 
common problems 24 hours a day. 

One of OTAN’s goals was to develop as much technical expertise as possible at the local 
agency level, so that subscribers would be able to assist new and future users. While the 
training and technical assistance that OTAN offered were accomplishing this to a certain 
extent, the staff created Online Academy workshops to further increase local agencies’ 
capacity to provide assistance in their regions. The 60 hands-on Academy workshops that 
OTAN conducted were provided to nearly 1,000 users. OTAN also continued to use statewide 
conferences to promote itself. OTAN’s staff gave 28 presentations throughout the state, 
reaching an additional 1,000 individuals. 

In addition to the Online Academy and general OTAN training sessions, OTAN identified 
three special populations that required targeted staff development: 



• Round Table Work Group Managers. OTAN designed training and reference materials 
specifically for the subscribers responsible for managing and leading special interest Work 
Groups. Examples of Work Groups included LAES (Latino Adult Education Services) 
and SCLLN (Southern California Library Literacy Network). OTAN’s staff provided 
customized training to 23 work group managers and offered training and assistance to 
representatives from 30 new adult education work groups; 

• Regional Resource Managers. Since part of the Resource Centers’ mission was to provide 
training and assistance to practitioners for using the OTAN Online communication 
system, OTAN conducted training for the managers to enable them to provide assistance; 
and 
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• Section 321 Program Administrators. After consulting with the CDE, the Staff 

Development Institute (SDI), and others, OTAN’s staff identified training needs particular 
to administrators. To address these needs, they developed six multimedia training 
packages for administrators. Topics included “Using the Adult Education Handbook,” (a 
guidebook to help administrators understand and comply with state regulations) “Funding 
Technology,” “Making Connections,” “Special Education,” “Technology and 
Leadership,” “World Wide Web,” and “Workforce Online.” 

Information and Reference Services. OTAN maintained relationships with the agencies, 
projects, clearinghouses, professional associations, and organizations that comprised its 
California and national informational network. In addition, OTAN’s staff attended 44 
conferences and monitored 150 print newsletters. It also contracted with Frost, Davis, & 
Donnelly to provide updates on adult education-related legislative activities. The information 
collected through these various sources helped to maintain and enhance the OTAN Forum, 
which increased in size from 35 MB to 5 1 MB during this three-year period. The WAIS 
keyword search engine implemented on OTAN WWW allowed users to more easily navigate 
this growing body of information. 

OTAN’s California Adult Education Archives also continued to grow. Several more materials 
were cataloged, bringing the total to 666 items. A California Adult Education History Web 
site was established, giving subscribers the ability to search the Archives; view vignettes from 
an adult education video; view portions of a CDE adult education history report; and access 
information about the California Adult Education Oral History Project. 

In addition to the electronic resources that OTAN maintained in its Forum and Archives, it 
added 97 links on its Web pages to other adult education Web resources. Links to hundreds of 
other electronic information sources were imbedded in documents provided on OTAN WWW. 

OTAN continued its relationship with the Regional Resource Centers, providing centralized 
acquisitions, cataloging, and technical assistance. Each Center maintained a collection of 
approximately 600 items that were searchable via the OTAN Forum prior to 1996. When the 
Centers established their own WWW home pages in 1996, OTAN helped them to improve 
their access to the bibliographic records of the collections by creating an online searchable 
database. 

To further serve the information needs of practitioners and researchers, OTAN maintained two 
unique reference collections in its Sacramento office— the Adult Education Reference 
Collection and the Educational Technology Collection. By 1997, these collections contained 
956 and 1,293 items, respectively and were searchable on OTAN WWW. OTAN’s staff 
responded to more than 4,000 reference inquiries, about a quarter of which required significant 
research to fulfill. 
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Reference inquiries to OTAN usually related to finding information on the OTAN Forum. The 
following are illustrative examples of actual requests: 



• An adult school vice principal needed information on adult education services for women 
to prepare for testimony before a legislative committee; 

• A CDE consultant asked for copies of prior staff development survey forms used by 
projects in California before preparing a new form; 

• An adult school principal requested statistics on adult literacy in California; 

• A contractor with CDE needed information on the background of distance learning in 
adult education before proposing new initiatives; 

• A potential student requested help with the best way to use the Internet to locate 
vocational schools in Fresno; 

• A teacher asked for a list of software titles concerning students with learning disabilities; 

• A leadership project staff person needed references on telecommunications in ESL in 
preparation for writing an article; and 

• A teacher wanted information on portfolio assessment. 



California’s welfare reform activities led to a greater demand on adult literacy and job training 
programs to serve CalWORKs 8 clients. OTAN supported the circulation of curriculum and 
model program materials through the VESL Workplace Clearinghouse and Employment 
Training Network Library. The number of cataloged items in the VESL Workplace 
Clearinghouse nearly doubled during the three-year period from 151 to 284. Practitioners 
could access the Clearinghouse by using a catalog (in both print and electronic form) that 
OTAN updated twice a year. The Clearinghouse continued to be the sole distributor of non- 
commercial vocational ESL materials in California. CDE established the Employment 
Training Network (ETN) using federal JTPA funds, which provided assistance to JTPA-funded 
projects in California. OTAN assisted the ETN by developing and hosting its Web site, which 
included an online search engine for the ETN Library of job training resources. By 1997, the 
ETN Library featured a collection of about 2,000 books and 200 videos that were loaned to 
JTPA-funded agencies free of charge. 

Technology Planning and Implementation. Early in 1994, OTAN initiated a process of 
providing technology planning and implementation assistance to Section 321/326 agencies. To 
better understand their needs, OTAN conducted a survey of these agencies in 1995. It revealed 



8 CalWORKs js the California program under the federal Temporary Assistance to Needy Families (TANF). 
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that while most programs were using computers for instruction, administration, or 
communication, just 17 percent had developed a technology plan. Eighty percent of the 
respondents expressed a need for a technology plan. 

OTAN responded by providing a wide range of training and technical assistance to programs. 

A total of 142 seminars was held from 1994 to 1995, covering topics such as planning for the 
use of technology, selecting software, and integrating technology into the ESL or ABE 
classroom. 

By early 1996, OTAN’s staff realized that another critical issue for programs— computer networking- 
would require their technical assistance. They provided seminars on topics such as local area 
networks, Internet connectivity, and distance learning. The topic of distance learning sparked 
the interests of many practitioners, who subsequently requested OTAN’s assistance with 
Internet-based learning, home-based instruction with video, laptop check-out programs, and 
other uses of technology. 



Enhancing OTAN’s System (April 1997 - June 1999) 

This period featured significant shifts in policy that redefined OTAN and some of its roles. 
OTAN also conducted a variety of activities, including improvements and upgrades to the 
communications and information systems, the provision of additional training and technical 
assistance, further enhancement of the information and reference services, and outreach and 
assistance to support technology planning. Its activities were focused in three areas: 1) 
electronic communication, 2) information and reference services, and 3) instructional 
technology. 

Electronic Communication. Three important policy and programmatic changes were made 
during this period that redefined OTAN’s electronic communication system: 



• To encourage additional electronic communication, CDE decided to provide free Email 
accounts to local programs receiving Section 321/326 funding. The number of free 
accounts that a program received was related to the number of learners it served. This 
policy was implemented in July 1998; 

• CDE agreed to transition OTAN out of the business of providing both Internet access and 
Email. The decision was based on the recognition that local agencies had greatly 
enhanced their use of computer technology in recent years and that the vast majority had 
access to the Internet, either through their host institutions (community colleges, school 
districts, etc.) or private providers (America Online, Earthlink, etc.). The results of a 1998 
technology survey that OTAN conducted of California adult education programs 
supported this decision. Compared with the results from a similar 1995 California 



Abt Associates Inc. 



History and Development of OTAN 



11-14 



Connecting People, Policies, and Practices: Evaluation of OTAN 



technology survey, the 1998 findings suggested that newer multimedia computers were 
swiftly replacing older machines. Survey respondents reported that 62 percent of the 
computers in use in 1998 were either Power Macs (Apple) or IBM-compatible machines 
with Windows 95, and thus probably capable of accessing the Internet. The survey also 
found that 87 percent of respondents had Internet access (OTAN, 1998a); and 

• To provide a more efficient system with fewer pieces of software (OTAN Online software 
for electronic communications and Web browser software for the information and 
databases on OTAN WWW), OTAN decided to migrate all of its components to the 
OTAN Web site. Staff developed a “registration enhanced” Web site, which allowed 
users to obtain OTAN accounts and to use the site free of charge. 9 



Prior to implementing these changes, OTAN had focused on enhancing its existing systems. 
The OTAN Online Communication System software was upgraded in mid- 1998 to make it 
similar in appearance to the World Wide Web, in order to help long-time OTAN Online users 
transition to the Web. The upgrade also allowed some new features, including threaded 
discussions. World Wide Web homepage creation, and spell check. 

The open registration system attracted a large number of new users. According to OTAN staff, 
as of January 1, 1999 (prior to the implementation of the new system), there were 1,001 
accounts-with between 20 and 30 percent being inactive. By mid-May 1999, staff reported 
that 689 of those accounts had logged onto the new system, representing about 69 percent or 
virtually all of the estimated number of active accounts. By the end of June 1999, OTAN had 
obtained more than 1,400 new members, bringing the total to 2,1 19 (OTAN, 1999). OTAN 
staffs analysis of these new registration records revealed that most subscribers were California 
adult education teachers and most accessed the Web using America Online as their Internet 
Service Provider (ISP). This 100 percent increase occurred with OTAN conducting minimal 
publicity outside the group of existing subscribers. 

Access to the Web site also increased, with the number of hits per month growing from 
453,000 hits during December 1998 (just prior to the implementation of the open registration 
site) to 713,000 hits for the month of May 1999— an increase of about 57 percent. In Appendix 
B, Exhibit B-l shows the rising number of hits and user sessions 10 over the 12-month period of 
June 1998-May 1999. A comparison of May 1999 data to that of March 1997 (the close of the 



9 The redesigned Web site was released in January 1999, In addition to the no-cost registration system, it has a number 
of other new features, including an online User Guide to assist inexperienced visitors, a new Java-based pull-down 
menu system, a new home page, and a redesigned Round Table area, 

10 A hit is an action on the Web site, such as when a user views a page or downloads a file. A user session is a session of 
activity (all hits) for one visitor to the Web site. By default, a user session is terminated when a user falls inactive for 
more than 30 minutes. 
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last contract period) reveals an even larger increase. The number of Web site hits rose from 
229,000 in March 1997 to 713,000 in May 1999— an increase of more than 300 percent. 

OTAN’s decision to discontinue its roles as an ISP and Email provider as of June 30, 1999 
resulted in an increased access from users with non-OTAN addresses. Presented in Exhibits 
B-2 to B-6, Appendix B are data on the numbers of hits and user sessions from the three most 
active providers: OTAN, K-12 institutions in California, and America Online (AOL). The 
level of access by users with California school district addresses (kl2.ca.us) rose from 51,000 
hits in June 1998 to 1 1 1,882 hits in May 1999, and the level of access by users with AOL 
accounts (aol.com) increased from 22,000 hits in June 1998 to 42,000 hits in May 1999. AOL 
users accounted for a large number of user sessions (28 percent of all user sessions in May 
1999), but a relatively smaller number of hits (six percent of all hits in May 1999), suggesting 
that they visited OTAN in large numbers but engaged in less activity relative to users with K- 
12 California and OTAN addresses. 

Another priority for OTAN was to assist other agencies gain a presence on the Web. Thirty- 
five programs developed their own Web sites using the online “Web Generator” feature and, as 
of September 1998, OTAN had provided links to the Web sites of 44 various education and 
library entities (OTAN, 1997b; OTAN, 1998b; OTAN, 1998c; OTAN, 1999). In addition, 
OTAN hosted the Web sites of several agencies on its servers and supported those sites with 
development and maintenance assistance. One of the sites hosted by OTAN— CNN San 
Francisco— was a partnership among OTAN, the California Distance Learning Project (CDLP), 
and CNN. It featured engaging news stories that could be used in literacy instruction. 

During this period, OTAN conducted focus groups and surveys to determine the types of 
training and assistance that subscribers needed. The results led OTAN to offer 97 training 
workshops across California, reaching 2,207 individuals between April 1997 and June 1999 
(OTAN, 1997b; OTAN, 1998b; OTAN, 1998c; OTAN, 1999). Session topics included: 
“OTAN Web Site Email and Collaboration,” “OTAN Web Site Content and Searching,” 
“Beginning Web,” “Advanced Web,” and “Basic OTAN Online.” In addition, OTAN 
developed a variety of supplementary materials to answer commonly asked questions, 
highlight new features on OTAN Online and WWW, and meet the needs of specific 
populations. All materials, including those used during training sessions, were made available 
in PDF format on OTAN’s Web site. Examples of these materials include OTAN’s quarterly 
newsletters and a guide for using the A22 Course Approval System. 

OTAN continued to provide free and unlimited technical support to its subscribers. It 
responded to 2,337 telephone calls during this period (OTAN, 1997b; OTAN, 1998b; OTAN, 
1998c; OTAN, 1999). Staff also responded to emailed questions and continued to maintain the 
Help Desk, a database of solutions to common problems accessible through the Web site 24 
hours a day. OTAN’s staff conducted a Client Satisfaction Survey in early 1999 and asked 
subscribers to describe the assistance they needed. Connection-related issues (modem 
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problems, finding an Internet Service Provider, Firewall/proxy server problems) generally were 
the types of assistance the subscribers identified. While connection issues were not directly 
under OTAN’s control, staff provided support since many subscribers did not have easy access 
to technical assistance in their local agencies. 

Information and Reference Services. OTAN’s staff maintained relationships with the local, 
state, and federal agencies and organizations with which it shared information. The Forum 
was updated regularly and was expanded to nearly 20,000 files and database records, 
representing 404 MB of information as of June 1999 (OTAN, 1997b; OTAN, 1998b; OTAN, 
1998c; OTAN, 1999). Specific improvements to the Forum during this two-year period 
included: 



• The “For Teachers” area was enhanced to include a monthly feature integrating the 
Internet into instruction, and a comparison of Integrated Learning Systems; 

• The Who’s Who section was expanded to include a searchable database of key state and 
U.S. adult education contact persons; 

• The Adult Literacy Reference collection grew by 39 percent to a total of 1,325 cataloged 
items; 

• The VESLAVorkplace Clearinghouse grew by 19 percent to 338 cataloged items; 

• The Adult Education Archives collection increased by 44 percent to 958 cataloged items, 
and OTAN’s staff began developing policies and procedures for digitizing many of the 
archival materials; and 

• The Educational Technology Collection grew by 15 percent to 1,486 cataloged items. 



During the period from 1997 to 1999, staff responded to more than 3,300 reference inquiries 
on a variety of topics, including research on legislation, statistics to support grant proposals, 
information regarding staff development, and resources for program improvement. Responses 
to a Client Satisfaction Survey that OTAN conducted in early 1999 revealed a very high level 
of satisfaction with the reference assistance provided by OTAN staff, with respondents rating 
their overall satisfaction at 4.75 on a five-point scale, (one being “unsatisfied” and five being 
“completely satisfied”). 

As the Forum grew in size and complexity over the years, it became necessary to formalize the 
processes involved with the preparation of information for inclusion in the Forum. In 1992, 
OTAN developed its first Document Processing Manual, which delineated these policies and 
procedures. In a third edition of the manual released in 1998, OTAN provided information on 
topics such as scanning, formatting, titles, headers and footers, graphics, post 
formatting/information storage, file maintenance, and databases (Harris et al, 1998). Although 
the information that OTAN receives has come from a variety of sources (CDE, OTAN 
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subscribers, World Wide Web sites, etc.) and in a variety of formats (Email, paper documents, 
World Wide Web/HTML pages, word processing files, etc.), documents usually have been 
presented in one of three formats— text only, HTML, or PDF— depending on nature of the 
original document. 

Instructional Technology. In the area of instructional technology, OTAN continued its role 
in providing resources and technical assistance to support technology planning. During this 
period, OTAN offered technology planning services and technical assistance to more than 20 
agencies (OTAN, 1997b; OTAN, 1998b; OTAN, 1998c; OTAN, 1999). 

OTAN also conducted activities focused on helping adult education agencies implement and 
use technology, such as the following: 



• Established an online Adult Education and Technology Vendor Database. Accessible 
through OTAN’s Web site, the database assists adult educators make informed decisions 
by offering descriptions of adult education and technology products with links to vendor 
Web sites. The database is searchable by vendor, keyword, and type of product, and is 
updated continuously by OTAN staff; 

• Began a collaboration with CDLP and SDI to develop an electronic resource for adult 
educator staff development. The resulting project, called Teacher Training Online (TTO), 
provided self-paced, self-directed professional development via the World Wide Web. 

The first course offering focused on Family Literacy; 

• Developed three multimedia programs available through the Internet to help adult literacy 
instructors implement Web-based instruction. After consulting with local agencies to 
determine the topics that would be most useful, OTAN developed three programs: WWW 
Based Learning, Technology & Language Learning, and WWW & Workforce Online; 

• Continued its partnership with CNN San Francisco to host its Web site and provide 
support for the conversion of CNN news stories to literacy lessons; 

• Provided a variety of assistance— selection of software and equipment, provision of 
training and technical support— to help the Regional Resource Centers implement a 
videoconferencing system that would connect Center managers and improve 
communication; and 

• Updated the information about technology use in California by conducting a survey in 
1998 (similar to the 1995 survey) to identify existing levels of use and new needs in 
technology. 
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Summary 

Since its inception, OTAN’s staff have used state-of-the-art technology to develop an 
electronic communication and information infrastructure for serving adult educators. OTAN’s 
development has been both methodical and iterative, initially building a foundation for the 
infrastructure, then expanding the types of services and products available in its system, and 
most recently expanding the capacity and linkages of the system. What began as a special 
project to serve adult educators in California developed into a national resource for linking 
people and information electronically. 

Four features have characterized OTAN’s evolution: 



• Use of state-of-the-art technology to create and refine the communication system and 
provide services, including ongoing updates to the hardware and software that have 
allowed for expansion and efficiency; 

• Provision of training and technical assistance using a variety of modalities to support all 
levels of adult education program staff, state policymakers, and others in accessing and 
using OTAN; 

• Development of different communication vehicles for fostering information sharing, 
building collegial relationships, and improving adult education practices; and 

• Creation of an information base that includes the development of new materials, gathering 
of existing information, and the linking with other databases to provide resources for adult 
education program management, instruction, training, and policy development. 

In undertaking these activities, OTAN’s staff have worked to provide quality services in a 
timely manner that have enabled adult educators to access OTAN and have increased the 
capacity of adult educators in the state to use computers. The data collected by Abt Associates 
and by OTAN have indicated strong customer satisfaction with the assistance that OTAN has 
provided, as well as with OTAN’s gradual expansion of materials and assistance to support the 
use of the system. This satisfaction may be attributed both to the methodical approach that 
OTAN’s staff used in developing the system, as well as to the staffs attention to the need for 
ongoing system refinement, materials creation, and customer support. This approach has 
resulted in a system with a wealth of resources that can be used in a myriad of ways to improve 
adult education services. 
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III. Utilization of OTAN as a Communication 
Networkand Resource for Information 



In developing a communication network, OTAN has created and refined a variety of tools 
and products to promote the exchange of information and discussions among OTAN’s 
subscribers. The network was intended not only to assist local adult education staff in 
accessing the most current information concerning program regulations and requirements, but 
also to encourage interaction among program staff on operational and curricular issues. 
OTAN has served as a repository of research and programmatic information, as well as a 
gateway to national adult education resources available through the Internet. This chapter 
presents a discussion of the communication tools and information resources that OTAN has 
developed and the ways in which they have been used. 

Communication Network 

Description of Network Components. OTAN has developed various mechanisms for 
facilitating communication among its users, including Email and other components that 
evolved over time. With the redesign of OTAN’s Web site in January 1999, the Email 
function was no longer offered and the communication components were reorganized under 
the Round Table. In addition to the various components under the Round Table, OTAN 
implemented an electronic feature that expedites the California Department of Education’s 
adult education course approval system. 

Email. OTAN offered subscribers Email accounts from its inception through June 30, 1999, 
when it was discontinued. The decision to discontinue Email service was based on CDE’s 
and OTAN’s recognition that local agencies had greatly enhanced their use of computer 
technology in recent years and that the vast majority of OTAN subscribers had Email 
accounts and Internet access provided through their host agency (community college, school 
district, etc.) or a private Internet Service Provider (America Online, Earthlink, etc.). Abt 
Associates’ 1997-98 OTAN User Survey found that four out of five users (79 percent) had 
their own Email account in addition to the one that OTAN had provided. 

Round Table Areas. The Round Table areas consist of five components of services designed 
to facilitate discussion and sharing of information among OTAN subscribers. The Special 
Topic Workgroups are dedicated to particular topics or projects in which members can share 
information, pose questions, and exchange files. A variety of agencies operate workgroups 
for their members, as well as do representatives from special interest groups. Some 
workgroups are public, such that any OTAN subscriber can read and post messages. Other 
workgroups are private, with only members having access to them. An example is the 
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Southern California Library Literacy Network (SCLLN), a network of professionals from 
approximately 50 library programs. Since geographic location, time, and financial 
constraints prevent the group from meeting fact-to-face frequently, the workgroup allows the 
network members to share information and plan events. Each workgroup has a manager, 
who maintains a list of members and organizes members’ access to the workgroup. 

Other components of the Round Table are the OTAN Exchange, Live Chats, List Servers, 
and Want Ads. OTAN subscribers use the Exchange to communicate on a wide variety of 
topics, including curriculum, technology, compliance with the Americans with Disabilities 
Act, and assessment. Through the Live Chats component, OTAN subscribers are able to 
communicate “live” on a variety of topics in a private or public setting. The Email List 
Servers are discussion groups that focus on specific topics such as adult education, alternative 
learning, English as a second language, and literacy. The final component, the Want Ads, 
enables OTAN subscribers to advertise available positions in their agencies or to look for 
positions in other adult education agencies. 

CDE Course Approval (A22) System. Agencies receiving state funding must submit course 
approval request (A22) forms annually. In 1997, OTAN unveiled the A22 Course Approval 
System that streamlined the approval process by allowing subscribers to submit requests 
electronically to the Department of Education. Subscribers also can use the system to view a 
list of approved adult education courses statewide. 

Usage and Perceived Quality of Communication Features. Prior to its termination, Email 
was the most heavily used communication feature, used by 72 percent of respondents to Abt 
Associates’ 1997-98 User Survey. One-third (33 percent) reported using it once a day or 
more, and another 33 percent used it one to four times a week. Over three-quarters of the 
survey respondents reported finding Email “easy” or “very easy” to use, and just six percent 
found it “difficult” or “very difficult.” The vast majority (93 percent) of the respondents 
reported that communicating with other programs is a primary reason for using Email. Other 
frequently noted uses included “improve program” (44 percent), “manage program” (39 
percent), and “working with others on a joint project” (35 percent). 

The Round Table area was accessed by about half of the 1997-98 User Survey respondents, 
and the Exchange was the most popular component, used by 42 percent of respondents. Of 
the respondents that had used Round Table, most (64 percent) categorized its use as “very 
easy” or “easy,” and just five percent found it “difficult” or “very difficult.” When asked to 
explain why they used the Round Table area, the most frequently cited reasons were to 
“improve program” (44 percent), “improve self’ (32 percent), and “work with others on a 
joint project” (32 percent). 
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In early 1999, changes were made to the appearance and organization of Round Table area. 
Most notably, a split-screen frame was implemented that allows subscribers to 
simultaneously view a list of message titles on one side of the screen and the content of a 
selected message on the other. (Previously, users had to switch back and forth between 
screens to view a list of messages and the contents of individual messages.) Abt Associates’ 
1999 User Survey asked subscribers to evaluate the ease of use of the redesigned Round 
Table. Of those who had accessed it, 23 percent said it was “Very Easy” or “Easy,” 13 
percent felt that it was “Average,” and just 5 percent found it “Difficult” or “Very Difficult.” 

From the telephone interviews that we conducted with subscribers, we identified several 
examples of how they use OTAN’s communication features, including: 



An administrator had a question related to admissions policies and wait lists for 
popular classes. She posted her question to the Exchange, and in a few days, had 
numerous responses. Had OTAN not been available, she felt it might have taken 
weeks or months to obtain the same level of information; 

An instructor, wanting to converse with colleagues around the state about their 
use of instructional materials and technology, found the real-time conversations 
through OTAN’s Live Chats to be extremely useful. Because of her program’s 
budgetary limitations, she isn’t able to do this through long-distance phone calls. 
Thus OTAN is the only vehicle that enables her to have this type of interaction; 
and 

An administrator reported using the Exchange regularly when he has a question 
about state compliance issues. For example, he posted a question related to the 
establishment of a committee to oversee the administration of one-time site 
grants. He received a response and followed up to obtain additional information 
via Email. 



Information Storage and Retrieval 

Description of Information Features. Prior to January 1999, most of OTAN’s information 
resources were organized within the Forum, a collection of electronic documents and 
databases that contained more than 20,000 records and documents. With the redesign of the 
Web site, these documents were placed under OTAN Resources, which consists of the 
following components: 
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California Adult Education History— a collection of various historical resources 
and projects that allows users to search the Adult Education Archives, which 
contains more than 1,000 items; 

CA Adult Ed Products— a searchable collection of products (videos, instructional 
materials, reports, etc.) that OTAN maintains and distributes on a cost-recovery 
basis; 

CDE Info— notices, documents, and other information from the California 
Department of Education; 

Codes and Regulations— excerpts of selected state and federal laws; 

Document Library— a collection of electronic documents (in text format) 
organized by more than 30 topics; 

Educational Grants— summaries of grant opportunities for adult education 
providers, accessible by keyword search or by browsing all grant postings; 

For Teachers— a collection of resources for instructors, including Course Outlines, 
Lesson Plans, and links to various resources; 

Legislative Information— primarily state-level, but including federal resources to 
help members track and understand legislation related to adult education, welfare 
reform, immigration, and other topics; 

Master Calendar— a list of activities and important dates, searchable by date, 
category, geographic region, and organization; 

Presentations— the presentations that staff from OTAN and the California 
Department of Education made at conferences and meetings, which members can 
download in PDF format; 

Reference Libraries— searchable databases that allow members to retrieve citations 
and other references to the Adult Ed Reference Collection, VESL/Workplace 
Clearinghouse, Employment Training Library, Adult Education Archives, and 
Educational Technology Collection; and 

Who’s Who— a searchable directory of key individuals in adult education, 
including OTAN members, California state officials, and contacts in the federal 
and other state governments. 



In addition to the components under OTAN Resources, other informational resources that can 
be accessed through OTAN are: 

• OTAN News— Links to current and archived announcements and stories pertaining 

to adult education; 
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• OTAN for Teachers— With the revision of the Web site, OTAN’s staff created a 
new area specifically for teachers that has three sections: Technology, 
Communication, and Resources. The Technology section contains general 
Internet information, assistance with technology planning, information about 
distance education, as well as hardware and software resources. Through the 
Communication section, OTAN subscribers can link with other sections of 
OTAN, utilize various Internet resources, provide feedback to the OTAN staff, 
and post ideas on the Teacher Exchange. The Resources section contains 
information related to teacher credentialing, professional development, career 
planning, as well as links to lesson plans and assessment information. Information 
about vendors also is included in this section; 

• Adult Education Links— OTAN maintains links to over 100 Web sites that are 
related to adult education instruction, administration, grant seeking, research, 
legislation, staff development, and other topics. OTAN subscribers can search for 
the Web sites that are categorized under more than 50 topics; and 

• Partner Sites-OTAN provides links to adult programs in the state, professional 
organizations, and adult education-related projects and agencies. 

Usage and Perceived Quality of Information Features. Abt Associates’ two surveys and 
interviews with OTAN subscribers indicated high levels of satisfaction with OTAN’s 
informational resources but differences in the extent to which subscribers used the resources. 
While most subscribers found the major informational components of OTAN to be easy to 
use, some areas were utilized more than others. 

Eight out of ten respondents to the 1997-98 OTAN User Survey conducted by Abt Associates 
indicated that they had used the News area, which at the time represented the highest level of 
use of any of OTAN’s Online components. In a subsequent survey and interviews, OTAN 
subscribers consistently mentioned the News area as one of the most important and useful 
features of OTAN because of its ability to synthesize a variety of information relevant to 
adult educators. 

When subscribers were asked to identify the Forum (the title of the area in OTAN before it 
was changed to Resources) components they found useful in the 1999 User Survey that Abt 
Associates conducted, more than half of the respondents identified four areas (see Exhibit HI- 
1). These were: CDE Information (81 percent), Legislative Information (66 percent), 
Educational Grants (57 percent), and Master Calendar (5 1 percent). These results were 
similar to the 1997-98 User Survey, which asked subscribers to identify which resource 
components they use. Three of the top four areas were the same— CDE Information (67 
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percent). Legislative Information (64 percent). Course Outlines (53 percent), and Educational 
Grants (52 percent). 



Exhibit 111-1: Use of Forum Areas 





Reported use of Forum Areas 




1997-98 User Survey 
(n = 252) 


1999 User Survey 
(n = 176) 


Response 


Frequency 


Percent 1 


Frequency 


Percent 1 


CDE Info 


169 


67 


142 


81 


Legislative Information 


161 


64 


117 


. 66 


Educational Grants 


132 


52 


100 


57 


Master Calendar 


101 


40 


90 


51 


Codes and Regulations 


109 


43 


83 


47 


For Teachers (including Course 
Outlines 

& Lesson Plans) 2 






71 


40 


Course Outlines 2 


133 


53 


- 


- 


Lesson Plans 2 


106 


42 


- 


- 


Document Library 


108 


43 


65 


37 


Presentations 2 


- 


- 


51 


29 


Reference Libraries 


64 


25 


51 


29 


CA Adult Ed Products 2 


- 


- 


42 


24 


Who’s Who 


86 


34 


38 


22 


CA Adult Ed History 2 


- 


- 


22 


13 



Sources: 1997-98 User Survey; 1999 User Survey conducted by Abt Associates Inc. 

1 Percents are based on the number of survey respondents (n=252 for the 1997-98 User Survey; n=176 for the 1999 User 



Survey). Respondents could select more than one answer. 

2 The 1997-98 User Survey did not directly ask respondents about their use of the “For Teachers” area, the Presentations 
area, the CA Adult Ed Products, or the CA Adult Ed History area. However, it did ask about use of specific “For 
Teachers” components — Course Outlines and Lesson Plans. 



In the interviews that we conducted with subscribers and in the analysis of the User survey 
responses, we identified examples of how subscribers had used OTAN’s informational 
resources, such as the following: 
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An administrator from an adult program needed to revise its citizenship 
curriculum. Initially, she tried to contact a few vendors directly, but felt that their 
products were neither relevant nor of sufficient quality. She had greater success 
with OTAN. After posting queries on the Exchange, she received lists of 
materials that other programs had found satisfactory. Through using Course 
Outlines, Lesson Plans, and the Vendor database, she found information on a 
number of other promising instructional materials and methods; 

Aside from the value of accessing OTAN’s information, a number of subscribers 
commented that they felt less isolated because of the way OTAN connected them 
with news and resources. A number of administrators commented that keeping 
track of legislation and new developments the field of adult education was central 
to their jobs, and OTAN was their primary vehicle for doing so; and 

Several administrators indicated that they had secured grants as a result of 
applying for grants they had learned about in OTAN’s Educational Grants section. 
Others have used the News postings and CDE Information areas to identify 
sources of public funds, including applications, deadlines, and requirements. 



OTAN as a Resource for Program Improvement 

Although program improvement was not an explicit goal of OTAN when it was created, 
subscribers had begun to utilize OTAN for this purpose. In our interviews with program 
administrators, all but two of the 18 participants indicated that they were involved in some 
type of program improvement activity, such as revising curriculum, implementing new 
assessments, investigating new instructional strategies, or making changes to program 
policies and procedures. They all indicated that they were using OTAN as a resource. About 
half of the administrators involved in program improvement activities were focused on 
revising curriculum, and they reported using OTAN in two ways: 1) accessing areas such as 
Course Outlines, Lesson Plans, or the LAES Round Table to identify the types of materials 
others were using; and 2) using the Exchange or other Round Table areas to pose specific 
questions to other practitioners about their use of different instructional materials and 
methods. Another frequently mentioned type of program improvement activity was 
identifying how technology can be used to facilitate student learning. Again, respondents 
used OTAN as a source of information (e.g., searching the Adult Ed and Technology Vendor 
Database) and as a communication vehicle (e.g., asking other practitioners about technology 
they have used through the Exchange or Round Table). 
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In addition to conducting telephone interviews, we posted a question on OTAN’s main page 
that asked members to describe how they had used OTAN to support program improvement 
activities. Our analysis of the interviews and the responses to the online question included a 
wide variety of activities. Examples of the responses, organized into three categories 
(Curriculum, Staff Development, and Program Management), are listed below. 

Developing or Revising Curricula and Identifying Teaching Materials: 

• “We’ve checked out course outlines available before re-inventing the wheel.” 

• “Very valuable. As someone new to California, this gives me insights into 
California’s programs. I print lesson plans to remind/encourage staff to teach our 
adult D.D. clients by breaking new lessons into steps for more success.” 

• “Our individualized H.S. diploma program has begun a major curricular 
enhancement. We have started with Fine Arts. By viewing course outlines on 
OTAN we discovered a fresh way of dividing our two levels of Fine Arts. Like 
[another program that had posted instructional materials on OTAN], we will 
divide them into performing and visual arts.” 

• “We have used OTAN sample lesson plans for our volunteers to get ideas about 
what to teach. We have also used some articles in the Document Library for 
volunteers with specific problems, such as how to teach ESL to someone who is 
learning disabled.” 

• “I’ve used sample parent ed course outlines from OTAN for content ideas for [a] 
new parenting class.” 

• “I am in the process of working on our CAI (computer-assisted instruction) and 
find the Vendor Database helpful.” 

• “I have adapted the course outlines for reading instruction and writing instruction 
... to use in my Tutor Training Courses. These materials help the tutors better 
‘picture’ a logical sequence of instruction. I am also responsible for Computer 
Literacy instruction and have adapted OTAN’s fine Internet training as part of the 
materials that I use. I design and teach computer literacy to older adults. I have 
used the resources in the documents library as a starting point for my research into 

, the area. When I am looking for information, OTAN is always my first stop.” 



Providing Staff Development or Enhancing Own Skills: 

• “Located training sessions for staff.” 

• “Reading the posted info has given me much insight. It’s like having many 
mentors at once.” 
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“Use the announcements of workshops, conferences, and teleconferences to 
distribute to staff.” 

“I’ve used the document libraries for research for papers required in credential 
classes.” 

“This Web site has been very valuable in my development and understanding of 
the adult education and ABE areas. I’ve spent quite a bit of time doing research 
and ‘fact finding’ in OTAN. Really like the Round Table.” 

“Since our teachers are at remote sights around the state it is an essential tool for 
staff development.” 

“OTAN is invaluable for personal/professional growth — keeping up with rapid 
changes. (Example: Workforce Investment Act).” 

“I gave an Internet workshop to teachers last fall. I used OTAN as my primary 
source.” 

“I use the calendar to keep posted on staff development offerings and finding 
information on meetings of interest and importance to me.” 



Improving Program Management: 

• “OTAN is super for course perusal and approval. Little paperwork, no mess or 
wasted time.” 

• “Asking about specific database software from colleagues.” 

• “This has been tremendously helpful ! ! ! I like to use it to get feedback on how 
others are doing things, are they having similar difficulties, and so forth. This was 
instrumental in obtaining info that I used to convince my district that our adult 
school was entitled to the Site Block Grant.” 

• “OTAN is the most expedient distribution method for new policies, gathering 
data, or improving communication.” 

• “I appreciate being able to electronically file reports.” 

• “We received a $40,000 grant from a source we found on OTAN.” 

• “OTAN has been a very important source of ‘leading edge’ information. I use it 

to be on top of pending and new legislation, grant opportunities, model programs 
and practices, and networking with other professionals. OTAN helps identify 
where new programs and methodology are taking place — we are particularly 
interested in integrating technology into curriculum and instruction.” 
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Barriers to Use 

The two surveys that we conducted in 1997-98 and 1999 revealed that levels of use vary 
among subscribers, from those who use OTAN on a daily basis to others who access it once 
every several months. This section examines factors that may explain subscribers’ frequency 
of use of OTAN. Also discussed are our reviews of two of OTAN’s resources— Course 
Outlines and Lesson Plans-as examples of the types of information that are available to 
subscribers. 

Barriers to Use Reported by Subscribers. The 1997-98 User Survey conducted by Abt 
Associates asked respondents to identify barriers they had encountered in using the OTAN 
Online Communication system. In Exhibit HI-2, the responses to this open-ended question 
are compared with those from a similar question on the 1997-98 Non-user Survey that we 
conducted, which asked respondents to provide reasons why they had not been using OTAN 
Online. One key difference between users and non-users appears to be technical capacity. 
Forty-three percent of the Non-user Survey respondents felt that the computer equipment they 
had to work with was either insufficient or completely lacking, compared with 16 percent of 
User Survey respondents. 

Other barriers listed by respondents on both the User and Non-user Surveys include a lack of 
time (26 percent of users, 24 percent of non-users) and dissatisfaction with OTAN’s 
content/organization (20 percent of users, 25 percent of non-users). 

The 1999 User Survey that we conducted produced similar findings. Subscribers were asked 
to compare their use of OTAN in early 1999 (after a redesigned version of the Web site was 
released in January 1999) with their use in Fall 1998. Twenty-four percent of respondents 
indicated that they were using it less often than they did in Fall 1998 or hadn’t used it at all in 
1999. When asked to explain why, 40 percent of those responding cited lack of time and 
local technical problems as the primary reasons for lack of use. Their responses are 
summarized in Exhibit III-3. 

The most commonly cited barriers to using OTAN-inadequate computer equipment and 
other local technical problems faced by subscribers-are not issues that OTAN can address 
directly. OTAN’s staff have recognized that the adult education field has varying technical 
capacities and types of software in use. They have designed the Web site accordingly, so that 
it will work with older as well as newer versions of Web browsers and will not require an 
excessive amount of computer memory. OTAN’s policy has been to ensure that the Web site 
is fully accessible to anyone using Web browser software one version older than the current 
standard. 
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Exhibit MI-2: Barriers to Use (Combined Users & Non-users) 



Response 


Total - 


Users & 


Responses of 


Responses of 




Nonusers (n=250) 


Users (n=153) 


Nonusers (n=97) 




Freq. 


Percent 


Freq. 


Percent 


Freq. 


Percent 


Barriers to use of OTAN: * 














Technical problems at 
subscriber’s site / inadequate 
computer equipment 


66 


26 


24 


16 


42 


43 


No time 


63 


25 


40 


26 


23 


24 


Inadequate OTAN content 
and/or organization of content 


54 


22 


30 


20 


24 


25 


Technical problems with 
OTAN 


49 


20 


42 


27 


7 


7 


Lack of training or knowledge 
of computers 


40 


16 


28 


18 


12 


12 


Can get information from other 
sources (e.g. web, OTAN 
users) 


15 


6 






15 


15 


There are no barriers 


15 


6 


15 


10 


- 


- 


New to position 


8 


3 


- 


- 


8 


8 


Don’t have access to OTAN 


3 


1 


- 


- 


3 


3 


Not a valid response 


6 


2 


2 


1 


4 


4 



Sources: 1997-98 User Survey and 1997-98 Non-user Survey conducted by Abt Associates Inc. 

* The questions from both the User and Non-user Survey were open-ended, with 153 responding on the User survey, and 



97 responding on the Non-user survey. In both surveys, some respondents cited more than one reason. 



While the barriers regarding lack of time and local technical problems are not issues that 
OTAN’s staff could remedy directly, they could address the barrier concerning the content 
and organization of OTAN’s areas. In the two surveys and interviews that we conducted, 
respondents noted two ways in which the content and organization of OTAN might be 
improved: 1) the ease of identification of information due to vague menus categories (e.g.. 
Partners) or the specificity of the mail Search feature, and 2) the number of postings in areas 
such as Course Outlines and Lesson Plans. 
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Exhibit 111-3: Use of Redesigned Web Site in 1999 



Response 


Frequency 


Percent 


Please describe your use of OTAN since the redesigned web site 
design was released in mid-January 1999. 


(n = 176) 




I’m using it more often than I did in Fall 1998. 


34 


19 


I’m using it about as often as I did in Fall 1998. 


99 


56 


I’m using it less often than I did in Fall 1998. 


20 


11 


I haven’t used it at all since it was redesigned. 


22 


13 


No response 


1 


1 


If you haven’t used the web site or use it less often, please 
explain why. * 


(n = 38) 




Haven’t had time to use it 


9 


24 


Local technical problems (access speed, modem problems, 
inadequate hardware) make it too slow or impossible to use 


6 


16 


New site is more difficult to use; takes too long to find things 


5 


13 


Liked the old version of the web site better 


4 


11 


Don’t feel the need to use it as often 


4 


11 


Need training to use it 


2 


5 


Can’t offer explanation 


2 


5 


Email is not a part of the new system 


2 


5 


Prefer OTAN Online 


1 


3 


This survey prompted me to explore the redesigned site 


1 


3 


Have just recently gotten around to registering self 


1 


3 


Unsatisfied with content of web site; material not kept current 


1 


3 



Source: 1999 User Survey conducted by Abt Associates Inc. 



This was an open-ended question, with 38 of 176 individuals responding. 



Strategies for Addressing Barriers to Use. During the time period that we were collecting 
data for this evaluation, OTAN’s staff were redesigning various aspects of OTAN, including 
the organization and categorization of information on the Web site. After we conducted the 
User and Non-user surveys, we presented our findings to the staff, who incorporated this 
information into the redesign. As a result, OTAN’s staff addressed the barrier that subscribers 
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had identified in our surveys concerning the ease of identification of information. OTAN’s 
staff organized the areas of the Web site so that they are understandable and have labels that 
communicate their contents. These changes should result in a greater ease of use of the Web 
site. 

A second barrier to use that subscribers had identified was the number of postings in various 
areas such as Course Outlines and Lesson Plans. The results from our interviews with 
OTAN’s staff indicated that they had solicited these documents routinely and had made 
special attempts to encourage teachers to submit their course outlines and lesson plans to 
OTAN for posting. It appeared that while OTAN’s staff had made significant efforts to 
obtain these materials, their efforts alone were not sufficient. Rather, more encouragement 
by CDE staff or local program administrators may be needed to stimulate the submission of 
lesson plans and course outlines. 



Review of Course Outlines and Lesson Plans 

As part of its assessment of OTAN’s products for program improvement, Abt Associates 
reviewed a sample of items from two OTAN areas— Course Outlines and Lesson Plans— to 
explore the factors that might influence subscribers’ use of these areas. Both the organization 
and the content of these areas was examined. 

Course Outlines. Course outlines are found in the “Course Approval” section of OTAN and 
also can be accessed using the Search function. Available under ten broad subject headings 
or “authorized areas” of California adult school instruction (e.g., Elementary Basic Skills, 
Vocational Education), the course outlines provide instructional frameworks for teachers to 
use in planning and implementing their own courses. Components included in the course 
outlines are: goals and purposes of the course, performance objectives, instructional 
strategies, times of instruction (number of hours needed to cover each section of a course), 
evaluation (including description of measurable evaluation criteria), and conditions under 
which a student must repeat a course. 

While the guidelines for the course outlines are specific and uniform, the quality of the 
sample of outlines that we examined contained differing degrees of specificity. For example, 
an outline under the Adults with Disabilities section contained a minimal framework for 
creating a community access course for mentally limited learners. In this outline, the 
instructional strategies were not fully described and evaluation measures were not described 
specifically. In contrast, a second outline we reviewed contained a more detailed description 
of a competency-based basic skills course. In this example, the competencies were listed 
clearly, the instructional strategies were delineated and explained, and a six-component 
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evaluation plan was listed. A third outline of a basic skills course contained materials along 
with the outline, such as a chart that linked learning objectives to suggested learning 
activities. This chart was written in clear language that could be used by either a new or 
experienced adult education instructor. Outlines in other sections, particularly those in the 
vocational education section, delineated courses in detail. It may be the case that the 
certification requirements guiding vocational courses such as cosmetology and manicuring 
provide instructors with more information to develop detailed syllabi. 

One result of this variation in the amount of specific information presented in the course 
outlines is that they may be of differing utility to teachers with different backgrounds. While 
experienced teachers may need only a broad framework to use in developing a course, a less 
experienced may require detailed, specific information for course development. 

Lesson Plans. Lesson plans, which are found in the “OTAN For Teachers” section of 
OTAN, are organized into the authorized areas of adult school instruction in California and 
are divided by general subject within 1 1 major categories. Within each subject, the lesson 
plans are organized alphabetically by topic. Each lesson plan contains the following 
components: lesson objective, introduction /warm-up exercise, presentation or body of the 
lesson, guided practice, evaluation guidelines, and ideas for independent practice or lesson 
follow up. 

As with the course outlines, the specificity and quality of the lesson plans vary. In our review 
of a sample of lesson plans, we found differences in the amount of information presented in 
the plans. For example, a lesson plan concerning store-loitering situations that could 
potentially be encountered by a convenience store clerk consisted of a short outline for free- 
form discussion that was to be used in conjunction with the viewing of a video on the topic. 
Another lesson plan designed to teach students how to read classified advertising had a clear 
delineation of the framework and sequence of the lesson, but contained minimal information 
about materials to be used or the probable length of lesson. A third plan for an ESL lesson 
on grocery shopping indicated a detailed vocabulary lesson and a multi-sensory activity that 
would appear to be engaging to students. This plan, however, did not specify an evaluation 
or follow-up strategy for the lesson. 

Recommendations. While the quality and content of the course outlines and lesson plans are 
somewhat beyond the control of OTAN’s staff, some additional information might be 
requested from those submitting these documents to facilitate their use. Three 
recommendations that might be considered for enhancing the quality of the lesson plans and 
course outlines are as follows: 
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Revise the Lesson Plan format to include a section that describes the instructional 
materials (videos, books, etc.) required to implement the lesson; 

Include in Lesson Plans a section where the plan’s author can describe the 
thinking or theory that led her/him to create the lesson s/he did. For example, if 
an instructor has selected a phonics-based approach for the lesson, she/he could 
include a brief description of the rationale for this approach. This information 
would help instructors reviewing the lesson plan to understand the approach 
better; and 

Include a glossary of commonly used instructional terms in the “For Teachers” 
section, since the lesson plans and course outlines may include terms that are not 
familiar to all adult educators. 
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Training Provided by OTAN 

Description of Training. OTAN has provided a variety of training workshops to assist adult 
educators to use OTAN as well as to access resources on the Internet. The content of the 
training has changed over the years to reflect revisions to OTAN’s system as well as new 
technological developments. For example, when OTAN operated the dual Online and Web 
systems, its workshops were focused on installing and using OTAN Online software, using 
Web browser software, and navigating the Web. OTAN’s shift to a Web^based system 
necessitated the discontinuation of OTAN Online-related workshops and the creation of 
additional Web-related workshops. 

As of June 1999, OTAN offered five workshops. Each was three hours in duration. The 
titles of the workshops are listed below, followed by OTAN’s descriptions of these activities: 



Introduction to the World Wide Web and the OTAN Web Site. This 
workshop provides an overview of the Internet, the World Wide Web, and the 
OTAN Web site. Topics addressed include how to use a web browser and how to 
create bookmarks, as well as a basic overview of the OTAN Web site. 
Participants are given the opportunity to complete hands-on exercises; 

OTAN Web Site Content and Searching. This session addresses information 
resources on the OTAN Online Web site and effective strategies for searching the 
different documents and databases (Master Calendar, Reference Libraries, 
Vendors' Database). Included are structure, keyword, and fielded searching. 
Participants also are given the opportunity to complete hands-on exercises; 

OTAN Web Site Discussion Areas. In this workshop, participants are 
introduced to the areas of the OTAN Web site where they can collaborate with 
other adult educators in California and around the world. Areas include online 
workgroups, listserves, and web discussions. Participants also are given the 
opportunity to complete hands-on exercises; 

Downloading, Plug-ins, and File Compression. This workshop is for users who 
are familiar with the World Wide Web and want to learn the basics of 
downloading programs and other files from the Internet. Participants are given 
the opportunity to download and install two popular "plug-ins" during class— 
RealPlayer and the Adobe Acrobat Viewer. RealPlayer allows the user to view 
videos and listen to audio while using a web browser. Adobe Acrobat Viewer 
allows the user to view the many PDF documents that are posted on the World 
Wide Web, including documents located on the OTAN Web site. Other topics 
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addressed in the workshop may include locating downloadable files on the World 
Wide Web and how file compression works; and 

• New Features of the OTAN Web Site. This workshop is designed for 

experienced users on the OTAN Web site. Participants are introduced to the 
newest features of the OTAN Web site, and the class is updated as new features 
are added to the OTAN Web site. Participants also are given the opportunity to 
complete hands-on exercises. 

These workshops have been publicized primarily through the Training section of OTAN’s 
Web site. Each workshop has been offered multiple times a year at various locations 
throughout California. Until the Regional Resource Centers ceased operation in June 2000, 
training was provided at the Centers, which offered a variety of technical support and other 
services for adult education providers in California. OTAN workshops also have been hosted 
by local adult education agencies. OTAN staff also have traveled to an adult school offering 
to host a workshop, provided that the school has adequate facilities, (e.g., a computer lab with 
a fast Internet connection), and there is a sufficient number of educators (eight or more) 
interested in participating. Any adult educator can register for a workshop by submitting 
information through the Web site or by telephoning OTAN directly. OTAN has not charged 
a fee for the workshops, but some host sites have charged a small site fee— e.g., $2 per 
person— to defray costs associated with offering the training. Those interested can download 
the workshop handouts, which are available in PDF format on the Web site. 

Use and Perceived Quality of Training. In order to determine OTAN subscribers’ 
perceptions of the quality of the training that they have received, questions on this topic were 
included in Abt Associates’ 1997-98 OTAN User Survey. Over half (57 percent) of the 
respondents from the User Survey had participated in at least one training session. At the 
time the survey was conducted, workshops were focused on assisting subscribers in: 
installing and using OTAN Online, using Web browsers, and learning about relevant Web 
sites. Respondents reported high levels of satisfaction with the training. Nearly all survey 
respondents (99 percent) who had received training found it helpful. When asked to describe 
their satisfaction with the training and technical assistance, the survey respondents indicated 
that they had achieved their goals of learning how to use OTAN (42 percent); that the training 
was of high quality (33 percent); and that they had learned about the Intemet/WWW (12 
percent). Exhibit IV- 1 presents the types of training that users received and their assessments 
of the types of training that were most helpful. Exhibit IV-2 lists the factors those survey 
respondents identified as important in facilitating their continued use of OTAN. Almost half 
of the respondents cited their ability to access information as a key reason for accessing 
OTAN. 
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Exhibit IV-1: Types of Training Received by Users and Level of Satisfaction 



Response 


Types of training 
received * 


Which type of training 
was the most helpful? 


Frequency 
(n = 143) 


Percent 


Frequency 
(n= 109) 


Percent 


How to use OTAN Online 


136 


95 


49 


45 


Installation/setup of OTAN Online 


76 


53 


13 


12 


How to use the WWW 


70 


49 


10 


9 


How to use Netscape or Internet Explorer 


64 


45 


13 


12 


Education web sites 


36 


25 


3 


3 


Distance learning web sites 


28 


20 


0 


0 


Other 


9 


6 


21 


19 



Source: 1997-98 User Survey conducted by Abt Associates Inc. 

* Percents are based on the number of users that indicated they had received training (n=143). Some indicated 



participation in more than one type of training. 



Exhibit IV-2: Factors Facilitating Use of OTAN Online 



Response 


Frequency 


Percent 


Factors facilitating use of the OTAN Online communication system:* 
Ability to access information 


65 


42 


Ease of use 


41 


27 


Support from OTAN staff (tech support) 


37 


24 


Availability of training 


22 


14 


Ability to communicate with other programs, colleagues 


21 


14 


Availability of a computer 


9 


6 


Knowledge of computers and/or Internet 


8 


5 


Support from own staff 


5 


3 


Availability of free software 


5 


3 


Mandated for 321 organizations 


5 


3 


Availability of print support (the manual/guide, newsletter) 


3 


2 


Access to WWW 


3 


2 


Ability to access at home 


3 


2 


Non- valid response 


3 


2 



Source: 1997-98 User Survey conducted by Abt Associates Inc. 

* This was an open-ended question, with 1 53 of 252 individuals responding. Some gave multiple responses. 
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Subscribers who did not participate in OTAN’s training were asked to explain why they had 
not done so (see Exhibit IV-3). Forty-two percent of the respondents indicated logistical 
reasons as a barrier to participation, such as location and time. Other reasons respondents 
cited were that workshops were not held in areas convenient to users (30 percent); workshops 
were not offered at convenient times (12 percent); or that respondents felt that they had no 
time for training (12 percent). Additional reasons generally concerned subscribers’ existing 
comfort with OTAN and/or with computers in general. 

Subscribers who participated in training were more likely to use OTAN regularly. Of those 
categorized as heavy users in the 1997-98 User Survey, 64 percent had participated in OTAN 
training, compared with 30 percent of respondents who used OTAN infrequently. 



Exhibit IV-3: Reasons Users Have Not Received Training 



Response 


Frequency 


Percent 


Why have you not received any training? * 






Too great a distance away from training 


31 


30 


Training not offered at convenient times 


12 


12 


No time for training 


12 


12 


OTAN is easy enough to use 


12 


12 


Got training from another source 


12 


12 


Never felt the need for training 


11 


11 


Already an experienced user of computers and/or Internet 


10 


10 


Unaware that training existed 


7 


7 


New to position; wasn’t there when training was offered 


6 


6 


Users guide is sufficient 


2 


2 


Unaware that training was needed 


2 


2 


Training didn’t appear to meet needs 


• 2 


2 


Non- valid response 


4 


4 



Source: 1997-98 User Survey conducted by Abt Associates Inc. 



This was an open-ended question, with 103 of 252 individuals responding. Some gave more than one answer. 
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Technical Assistance Provided by OTAN 

Description of Technical Assistance. As of June 1999, OTAN offered five types of 
technical assistance, each of which was publicized in the Technical Assistance section of the 
Web site. The types of assistance were: 

• Telephone. A toll free number (800-894-3113) connects subscribers directly 
with an OTAN tech support staff person, available weekdays from 8:00 am until 
5:00 pm. The toll free number is only for California-based users. A local number 
(916-228-2580) also is available; 

• Email. Subscribers also may Email their questions to <support@otan.dni.us>; 

• Help Database. By entering keywords and describing the type of computer they 
use, subscribers can search an online database of solutions to common technical 
problems; 

• Tech Support Request Form. Available online is a short form that subscribers 
can complete which describes the type of computer they use and the nature of the 
problem or question that they have. Subscribers can choose to be contacted by 
OTAN staff either by phone or Email; and 

• Publications. OTAN’s Online Connection is a quarterly newsletter containing 
general information about OTAN, notices of changes to the system, highlights of 
areas/features, answers to frequently asked questions, and other information. The 
paper version is mailed to subscribers free-of-charge. The newsletter also is 
available online in PDF format along with other files that users can download for 
reference, such as handouts from OTAN training and the Web Sitemap. 

Use and Perceived Quality of Technical Assistance. Three out of four respondents to Abt 
Associates’ 1997-98 User Survey indicated that they had received some form of technical 
assistance. As indicated in the two surveys and in numerous telephone interviews that Abt 
Associates’ study team conducted with subscribers, the assistance provided by OTAN was 
one of the critical factors facilitating subscribers’ use of the system. Survey respondents 
indicated that one reason for their positive assessment was the overall quality of the 
assistance they had received. The respondents thought that OTAN’s staff were exceptionally 
capable and patient in answering a wide range of inquiries, and that their problems were 
solved in a timely manner. The variety of assistance options that were available to 
subscribers made obtaining help a relatively easy process, which our telephone interview 
respondents thought was in contrast to the sometimes unfriendly assistance offered by many 
Internet Service Providers, software companies, and hardware manufacturers. As indicated in 
Exhibit IV -4, the toll-free telephone support line that OTAN operated was rated by survey 
respondents as the most satisfactory means of technical assistance, with 96 percent rating it as 
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“satisfactory” or “very satisfactory.” In addition to providing telephone and Email assistance, 
OTAN staff have traveled to program sites to address problems that cannot be solved 
remotely. 

OTAN subscribers also thought that OTAN’s technical assistance was important because it 
often was the only technical support they could obtain for computer- or Internet-related 
issues. Since adult education programs— particularly those operating within community- 
based organizations— generally have limited resources for technology and related support, 
OTAN sometimes has been the only available source of assistance. Several of our telephone 
interview respondents indicated that without OTAN’s support, their computer-related 
problems would not have been resolved for long period of time, thus preventing their use of 
OTAN and the Internet. This was particularly true for those who were not facile in using 
computers when they began as subscribers to OTAN. Because the technical support was 
accessible and staff were patient, these subscribers felt comfortable asking basic questions. 

Exhibit IV-4: Levels of Use and Satisfaction with Technical Assistance 



Response 


OTAN Online 
Users Guide 


OTAN Online 
Newsletter 


Workshops & 
Conferences 


OTAN 

Presentations 




Freq. 


Per. 


Freq. 


Per. 


Freq. 


Per. 


Freq. 


Per. 


Level of satisfaction:* 


















Very Satisfied 


47 


34 


61 


40 


73 


45 


71 


50 


Satisfied 


72 


52 


73 


47 


71 


44 


50 


35 1 


Neutral 


20 


14 


19 


12 


17 


10 


19 


13 


Dissatisfied 


0 


0 


1 


1 


1 


1 


3 


2 


Total 


139 


100 


154 


100 


162 


100 


143 


100 



Response 


OTAN’s Toll Free 
Support 


Electronic 
Support Form 


Email Message to 
Support Area 


OTAN’S WWW 
Help Desk 




Freq. 


Per. 


Freq. 


Per. 


Freq. 


Per. 


Freq. 


Per. 


Level of satisfaction:* 


















Very Satisfied 


120 


73 


39 


47 


53 


47 


32 


40 


Satisfied 


38 


23 


19 


23 


34 


30 


25 


31 


Neutral 


5 


3 


24 


29 


24 


21 


22 


28 


Dissatisfied 


2 


1 


1 


1 


2 


2 


1 


1 


Total 


165 


100 


83 


100 


113 


100 


80 


100 



Source: 1997-98 User Survey conducted by Abt Associates Inc. 

* Level of satisfaction percents are based on the number of users of that particular support/resource, noted in the Total 
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Exhibit IV-5 lists the types of assistance sought by subscribers based on a survey that OTAN 
conducted in early 1999. Respondents reported that most of the assistance they needed was 
related to solving local connectivity problems, such as configuring a modem or 
troubleshooting problems with their organization’s firewall or proxy server 1 . Other common 
requests for assistance were related to using OTAN, such as retrieving forgotten passwords or 
locating a piece of information on the Web site. 



Exhibit IV-5: Types of Technical Assistance Sought by Subscribers 



Response 


Frequency 


Percent 


Please check what OTAN Technical Assistance helped you to do: * 






Troubleshoot a modem problem 


10 


30 


Resolve ID/password problems 


9 


27 


Other 


9 


27 


Interpret or explain an error message 


8 


24 


Download software 


6 


18 


Troubleshoot Firewall/Proxy Server problems 


6 


18. 


Locate information on the OTAN Web site 


5 


15 


Install your OTAN Online Software 


4 


12 


Install a Web browser 


2 


6 


Make software selections 


2 


6 


Make hardware selections 


1 


3 


Use a Web browser 


1 


3 


Find an ISP (Internet Service Provider) 


1 


3 



Source: 1999 Client Satisfaction Survey - Technical Assistance (conducted by OTAN) 

* Respondents could give more than one answer. Percents are based on an n of 33 individuals that responded to this 
survey. 
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l A firewall is a combination hardware and software buffer that many companies or organizations construct 
to protect their internal networks from intruders or hackers who might use the Internet to break into those 
systems. Proxy servers speed up web access by storing web pages on the local server. 
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OTAN’s Role with Special Projects 

Beyond its traditional activities of operating a communications and information network and 
providing training and technical assistance, OTAN has provided guidance and assistance to 
support special projects that serve targeted populations of adult educators and/or learners. 
OTAN has assisted these projects use technology to reach their goals. Some examples of 
past activities are: 

• The Southern California Library Literacy Network (SCLLN) consists of nearly 50 
library-based programs throughout southern California that offer basic reading 
and writing instruction to adults, in addition to providing other services. SCLLN 
began working with OTAN in 1996 to establish ways of using technology to 
remove geography as a barrier to collaboration and communication. SCLLN 
ensured that all member libraries had OTAN accounts and established a Round 
Table Work Group. According to SCLLN members interviewed by Abt 
Associates’ staff, the resources that OTAN provided— Email, chat rooms 

(for real-time meetings), and Round Table— have resulted in a greater amount of 
collaboration and joint ventures. OTAN also has hosted SCLLN’s Web site and 
an electronic version of “Tutor Exchange”— a newsletter for the 5,000 tutors 
volunteering in SCLLN libraries. One SCLLN member commented that OTAN 
has helped to “build our community” of library literacy providers; 

• The Latino Adult Education Services project (LAES) developed instructional 
materials for Latino ESL learners. It began in 1991 and received funding support 
from the California Department of Education as a National Literacy Act, Section 
353-funded project. LAES developed four categories of materials: advancing 
work and career; promoting community participation; managing family life; and 
developing life-long learning. LAES used OTAN as a resource for 
communication, research, and dissemination of the materials it developed. The 
OTAN librarian assisted with the research that informed the development of the 
materials, and other OTAN staff helped to establish and host a LAES Web site. A 
Round Table Work Group was created for LAES members, and OTAN staff 
assisted members in posting materials on the Work Group for quick retrieval. 
When LAES released its instructional materials through OTAN, subscribers used 
Email or the Exchange to pose questions or post comments for the developers and 
other users of the materials; and 



The California Email Project is a Web site that OTAN hosts which features 
different samples of adult ESL student writing (personal stories, recipes, etc.) 
from classes around the world. In addition to providing Web server space, OTAN 
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staff have provided many hours of technical assistance to construct, maintain, and 
improve the site. 



Summary 

OTAN’s staff have provided training and technical assistance that have been instrumental in 
enabling subscribers to feel comfortable with the use of technology and to become facile in 
accessing the various services and materials that comprise OTAN. A key characteristic of 
OTAN’s assistance has been the willingness of OTAN’s staff to explain the technical aspects 
of OTAN’s system and their availability in assisting subscribers in solving their technology- 
related problems. Not only have OTAN’s staff provided guidance related to the use of 
OTAN, they also have aided local adult education program staff in setting up computer 
systems and addressing general computer-use issues. 

To reach the variety of adult education programs located throughout the state, OTAN’s staff 
have traveled to various locations, provided training at the Regional Resources Centers, and 
arranged for training at local adult education programs. As reported in Abt Associates’ 
survey of OTAN users, their participation in training and receipt of technical assistance has 
influenced their high use of the OTAN system. 

OTAN also has facilitated the work of special projects supported by the California 
Department of Education’s Education Support System Division (formerly the Adult 
Education Educational Options and Safe Schools’ Division). These special projects have 
linked members of service provider groups, such as the library system, and have facilitated 
the work of curricular development projects. In this way, OTAN has helped individuals as 
well as groups of individuals involved in development and assistance projects. 
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This evaluation examined four aspects of the operation of OTAN: 1) the quantity and 
perceived quality of the services and materials that OTAN has produced; 2) the ways in 
which staff in local adult education agencies have utilized OTAN; 3) the effects of OTAN’s 
services on adult education program operations, staff, and policy development, and 4) the 
ways in which OTAN’s experience can inform further development of online services and 
products. The conclusions drawn from the evaluation are discussed in the contexts of the 
main components of service carried out by OTAN— electronic communication, information 
and referral services, and instructional technology. 

For the past decade, OTAN’s staff have implemented an electronic communication network 
that initially was intended for use by adult education administrators, instructors, program 
staff, state policymakers, and researchers in California and which became available to adult 
educators worldwide. The California Department of Education’s primary goal in supporting 
the development of OTAN was to provide a mechanism whereby adult educators in the state 
could share information, practices, and perspectives electronically to assist them in providing 
quality services. OTAN has met this goal though its iterative development of an electronic 
infrastructure for communicating, storing, and retrieving information. As the state of 
technology has advanced, OTAN has extended its system to incorporate new tools and 
information that have enabled the system to move beyond its original intent. Discussed 
below are our conclusions concerning the use of the OTAN system and possible future 
directions for OTAN as a state and national resource. 



OTAN’s Development and Implementation 

Phases of Development. OTAN’s development has been a phased process that began in 
1989. Since its inception, OTAN’s activities have helped to transform the state-of-practice in 
adult education in the state of California from limited or no access to computers for 
communication and information retrieval to high access. This transformation has occurred in 
three phases: 

• Building the Foundation: December 1989- January 1994. During this period, 
OTAN’s staff established an electronic network, provided staff development to local 
adult education agencies to access and use the network, identified and developed 
information resources for retrieval from the network, and conducted outreach to 
promote the use of OTAN among adult educators; 

• Building a Communication and Information System: February 1994-March 
1997. As the number of subscribers and the ways in which they used OTAN 
increased, OTAN’s staff focused on improving the quality of the technical system and 
the information stored on the system. A number of technical upgrades that staff made 
during this period resulted in a more powerful and easier-to-use online communica- 
tion and information system. Some of these upgrades expanded the types of 
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communication that subscribers could undertake, such as the installation of the Email 
list server software to host online Email discussions as well as software to provide 
online multimedia content. In addition to the provision of ongoing training and 
technical assistance, OTAN’s staff conducted targeted staff development to expand 
the capacity of key system users, such as adult education agency administrators and 
Regional Resource Center managers. The expansion of the library of online materials 
continued as did the establishment of linkages with other online systems. To enable 
local agencies to be more systematic in their use of technology, OTAN offered 
technology planning and implementation workshops to adult education providers. 

• Enhancing the OTAN System: April 1997- June 1999. During this period, OTAN’s 
staff made a number of changes in the system to respond to state policy shifts as well 
as subscribers’ needs. OTAN continued to improve the communication and 
information system, provided additional training and technical assistance, further 
enhanced the information and reference services, and offered outreach and assistance 
to support technology planning. Shifts in state policy prompted OTAN to adjust the 
electronic communication system in three ways: 1) to provide free email accounts to 
agencies receiving state adult education monies; 2) to transition out of the business of 
providing both Internet access and Email, because of the increased number of OTAN 
subscribers who no longer relied on OTAN for accessing Email; and 3) to refine 
OTAN’s Web site to include all components of service. Other new developments 
during this period included the piloting of online staff development in conjunction 
with the state’s staff development provider, the development of multimedia programs, 
and the expansion of partnership with entities such as CNN to provide online adult 
learner instruction. In January 1999, OTAN’s released the redesigned Web site. In 
addition to the no-cost registration feature, it has an online User Guide to assist 
inexperienced visitors, a Java-based pull-down menu system, a new home page, a 
redesigned Round Table area, and an expanded area called “OTAN For Teachers.” 

These phases of development represented a systematic approach to the creation of an online 
communication system that was responsive to users’ needs as well as incorporated the latest 
developments in technology. During each grant period, OTAN’s staff assessed the ways in 
which the system could be improved technologically as well as substantively and made the 
adjustments accordingly. The vision of OTAN’s director and staff enabled the system to 
keep pace with new developments without becoming too advanced for its intended users. 




Technical Assistance and Training. A critical element in the development of OTAN has 
been the training and technical assistance that staff created to support the dissemination of the 
system and encourage its use. The assistance that OTAN provided not only enabled local 
adult education program staff to become familiar with the online system and its capabilities, 
but also played a critical role in building the capacity of these staff to use computers. 

OTAN’s staff leveraged their training and technical assistance expertise by working with the 
Regional Resource Centers to deliver assistance to the adult education programs in their 
regions. OTAN’s staff also collaborated with these programs in hosting training sessions that 
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OTAN developed. As a result of this systematic approach to capacity building, the staff from 
OTAN and the Regional Resource Centers delivered training to approximately 20,000 
individuals during each of the three OTAN grant periods. As more adult educators became 
familiar with OTAN, the number of technical assistance calls addressed by OTAN’s staff 
increased such that during the third contract period, the staff responded to more than 25,000 
calls and reference inquires over this two-year period. 

Implementing and Using OTAN. As a result of OTAN’s training and technical assistance 
activities, its subscriber base grew substantially over the three grant periods. By 1999, OTAN 
had over 2,000 subscribers. A variety of factors influenced this growth, including the 
California Department of Education’s requirement that each grantee receiving state Section 
321 funding obtain an OTAN account, as well as the numerous dissemination activities that 
OTAN’s staff conducted. In this evaluation, the OTAN subscribers whom we surveyed and 
interviewed expressed high satisfaction with the training and technical assistance they 
received as well as with the resources they were able to access. The key barriers to use were 
the quality of the local technology and subscribers’ time to use the system. 

OTAN’s primary goal was to establish a communication systems that would link all 
stakeholders in the state’s adult education system. Our evaluation data substantiate OTAN’s 
achievement of this goal, particularly with regard to subscribers’ exchange of information 
concerning administrative and compliance issues. The California Department of Education 
has facilitated this exchange by increasingly using OTAN to communicate with its grantees 
and to process materials, such as the approval of courses and the applications for state 
funding. OTAN has become the platform for information on key issues affecting the 
operation of state-funded adult education agencies. 

As OTAN expanded its services, it’s role in promoting program improvement developed. In 
addition to posting course outlines and lesson plans created by California adult educators as 
well as research and library documents, OTAN became the gateway to instructional 
information available on the Internet. With the redesign of OTAN’s Web site in 1999, 
general resources as well as those targeted to teachers were better organized and more 
accessible to subscribers. By 1999, OTAN contained approximately 20,000 resource 
documents. To promote subscribers’ use of these resources, OTAN’s staff have highlighted 
on the Web site key resources and activities both within state and nationally that can assist 
program staff in developing and delivering quality instruction. 

While OTAN has great potential as a vehicle to support program improvement, this potential 
is still developing. In our interviews with program administrators, we found that many were 
involved in program improvement activities and had accessed OTAN’s resources to assist 
them. They also had used OTAN’s various communication vehicles to obtain assistance from 
colleagues and share information. Teachers’ use of OTAN for improvement purposes, 
however, was less prominent. While many teachers are self-motivated and have incorporated 
OTAN’s resources as a matter of practice, others may need external direction to prompt their 
use. 
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As a communication and information platform, OTAN has provided both resources and 
mechanisms for exchange among adult educators. The staff also have linked California’s 
adult educators to resources nationally to encourage the implementation of the best possible 
adult education services in the state. While OTAN can play a critical role in facilitating 
quality practice among adult educators, staff from the State Department of Education and 
local adult schools also need to be instrumental in fostering program improvement and 
OTAN’s use in this process. 



Future Directions 

The California Department of Education broke new ground in funding OTAN and it has been 
a unique service in the field of adult education. OTAN also served as a model for the 
National Institute of Literacy’s development of LINCS— the Literacy Information aNd 
Communication System. At this stage of adult education policy and practice in California 
and nationally, OTAN has potential in serving two key roles: as a platform to furthering the 
use of technology and as a gateway to adult education resources worldwide. 

Platform for Furthering the Use of Technology. OTAN has functioned as an information 
dissemination and communication base in addressing the needs of the California Department 
of Education and state-funded adult education programs. As these needs expand and as new 
developments in technology come online, OTAN has a key role in the following: 

• Expanding its services in facilitating the California Department of Education’s 
accountability and grants administration activities. As state-funded adult education 
programs submit their accountability data online, OTAN could promote the local use 
of these data for program management and improvement. By expanding its online 
training activities, OTAN could provide simulation exercises in using data that local 
program staff could access. OTAN also could stimulate subscribers’ exchange of 
information concerning the ways in which student data and other program information 
could be analyzed and used for program purposes. As the California Department of 
Education revises and enhances its grants management processes, OTAN has a key 
role in disseminating this information and serving as a mechanism for data collection; 
and 

• Disseminating new technology products funded by the U.S. Department of Education 
(ED). OTAN has been a collaborating partner in the development of the U.S. 
Department of Education-supported Cyberstep project. This project is one of three 
cutting-edge developments in online instruction, staff development, and assessment 
that ED has funded and which will be ready for national dissemination this year. As 
these products become available, there is a need to orient the field of adult education 
as to the ways in which these products can be incorporated into ongoing program 
services. OTAN is well-suited to serve as a platform for orienting adult education 
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practitioners to the use of these products, as well as to linking practitioners to other 
resources that could support their implementation. 

Gateway to Adult Education Resources. OTAN has a unique role to play in directing 
subscribers to the myriad of resources that can be accessed through its system. One critical 
use of these resources is in facilitating the delivery of quality adult education services. Two 
recommendations for OTAN’s role in program improvement are: 

• Revise the format for the submission of lesson plans to include sections regarding the 
teacher’s theory or guiding principles and related instructional materials. The 
inclusion of information about the theory or principles will help others to understand 
the developer’s instructional perspective in creating the lesson and how it might be 
applied. Information about related materials will guide others in selecting appropriate 
resources and applications that match the intent of the lesson; and 

• Use a problem-based method to stimulate adult education practitioners’ improvement 
of practice. Abt Associates’ work with the statewide program improvement initiative 
in Pennsylvania— Project EQuAL— demonstrated that adult education teachers and 
program administrators are motivated to address the operational problems that impede 
their delivery of quality program services. Using the problem identification process 
as a stimulant, staff are able to analyze information and use resources to try new 
practices. OTAN could adapt the problem-based technique by posting common 
problems that adult education practitioners face in managing and teaching as a way of 
engaging practitioners in program improvement. OTAN subscribers could be directed 
to the appropriate online resources and documents that relate to the problems that are 
posted. While many documents are not written to provide direct solutions to program 
issues, the information in the documents would stimulate ideas and new directions. 
Through OTAN’s Exchange and Chats functions, subscribers could share their 
solutions and suggest new practices. This process could serve as a starting point for 
adult education programs to use in addressing their operational issues. 

These recommendations are illustrative of the potential that OTAN has for enhancing its 
activities in facilitating communication and program improvement. OTAN has a critical 
function in disseminating information from the California Department of Education and in 
facilitating the state’s grants management process. Its role in obtaining documents and 
library resources also is expanding from a retrieval and storage function to a linking function. 
As a platform for technology, OTAN can direct subscribers to effective use and application 
of existing and new technological tools. As a gateway to information, it can guide subscribers 
in making informed choices regarding the variety of resources that are available to adult 
educators worldwide. 
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Appendix A 

Copies of the User Survey (1997-1998), Non-user Survey (1997-1998), 

and User Survey (1999) 



OTAN ONLINE/WWW SURVEY 



Name of Person Completing Survey: ; User ID: 

Name of Agency/Organization: 

Type of Agency/Organization: (Please check one.) 

□ K-12 Adult □ Community College □ Community Based Organization 

□ University □ County Office of Education □ Workplace Development Program 

□ Library □ ROP □ Other 

Address: 

City: State: Zip: 

Telephone: 



I. GENERAL INFORMATION 

1. Which category is the closest match to your job description? (Check one.) 

□ LEA/CBO Administrator □ Clerical or Other Support Staff 

□ Classroom Teacher □ State/Federal/Other 

□ Resource Teacher/Coordinator 



2. If you are a Classroom Teacher or Resource Teacher/Coordinator in adult education, 
please check the area in which you work: 



□ ABE/literacy 


□ Citizenship 




□ ESL 


□ Family Literacy 




□ Pre-GED 


□ Vocational Education 




□ GED/High School Completion 


□ ROP 




□ Other Adult Education 


□ Other (please list: 








) 


3a. How many hours per week do you work at 


your agency? 


hours 


3b. Is your job considered: □ part-time □full-time 




1 



69 



5 . 

6 . 



8a. 

8b. 



Do you use (please check one): 

□ OTAN Online 

□ OTAN World Wide Web (WWW) Site 

□ Both 

In what year did you begin using OTAN Online/WWW? 



19 



Please check the category that best describes the use of OTAN Online/WWW at your site 

□ I use it exclusively. 

□ I use it on behalf of another person. 

(Please check: supervisor colleague other). 

□ This is a shared ID with use by several staff members at my agency. 

□ This is an ID that allows open use by educators in my region, i.e.. Regional Resource 
Centers. 

□ Other (Please list: ) 

Please select who provides you with Internet access: 

□ OTAN 

□ District 

□ County 

□ Private account (List name of provider: ) 



Do you have your own email in addition to OTAN’s e-mail? 
Do you use OTAN’s email? □ Yes □ No 

If you answered yes above, please check how often you use it: 

□ More than once a day 

□ Once a day 

□ More than once a week 

□ Once a week 

□ Once or twice per month 

□ Less than once per month 



□ Yes 



□ No 
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II. TRAINING, TECHNICAL ASSISTANCE, AND SUPPORT SERVICES 

9. Have you ever used the technical assistance provided by OTAN? □ Yes □ No 

If yes, please check the methods that you have used to receive technical assistance and 
your level of satisfaction as defined in the Legend below: 

Legend: VS=Very Satisfied S=Satisfied N=Neutral D=Dissatisfied 

VD=Very Dissatisfied 



(If you don’t use a method, please check “DU” at the end of each row.) 



OTAN Online Users Guide: 


□ 


VS 


□ 


s 


□ 


N 


□ 


D 


□ 


VD 


□ 


DU 


OTAN Online Newsletter: 


□ 


VS 


□ 


s 


□ 


N 


□ 


D 


□ 


VD 


□ 


DU 


Workshops and Conferences: 


□ 


vs 


□ 


s 


□ 


N 


□ 


D 


□ 


VD 


□ 


DU 


OTAN Presentations: 


□ 


vs 


□ 


s 


□ 


N 


□ 


D 


□ 


VD 


□ 


DU 


OTAN’s Toll Free Phone Support: 


□ 


vs 


□ 


s 


□ 


N 


□ 


D 


□ 


VD 


□ 


DU 


OTAN’s Electronic Support Form: 


□ 


vs 


□ 


s 


□ 


N 


□ 


D 


□ 


VD 


□ 


DU 


Email Message to Support Area: 


□ 


vs 


□ 


s 


□ 


N 


□ 


D 


□ 


VD 


□ 


DU 


OTAN’s WWW Help Desk: 


□ 


vs 


□ 


s 


□ 


N 


□ 


D 


□ 


VD 


□ 


DU 



10a. Have you received training to use the OTAN Online communication system or the OTAN 
World Wide Web (WWW) Site? □ Yes □ No 

10b. If no, why have you not received any training? Please describe. 



10c.. If yes, please check the types of training you have received. 

□ Installation/Setup of OTAN Online 

□ How To Use OTAN Online 

□ How to Use the WWW 

□ How To Use Netscape or Internet Explorer 

□ Distance Learning Web Sites 

□ Education Web Sites 

□ Other (Please describe: 



lOd. Did you find the training helpful? □ Yes □ No 
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1 la. Which training and technical assistance from the list in Q. 10c have you found to be the 
most helpful? 



1 lb. Please describe why this has been helpful. 



12. In general, how would you characterize your need for additional training to make full use 
of the OTAN Online communication system? (Check one.) 

□ I need training to get started using the system. 

□ I need training on the advanced features. 

□ I occasionally need help to solve problems. 

□ I am satisfied with my present level of expertise using the OTAN Online system. 
Comments: 



13. Are there any types of training and technical assistance not offered that you would 
recommend? 



14. Please indicate how likely you are to participate in the following future OTAN support or 
training activities if they are available. Check a response for each item as defined in the 
Legend below. 

Legend: VL=Very Likely L=Likely M=Maybe PN=Probably Not N=Definitely Not 



Attend OTAN workshops or conferences: 


□ VL 


□ L 


□ M 


□ PN 


□ 


DN 


Attend OTAN Online “hands-on” training 
sessions at a location in your area: 


□ VL 


□ L 


□ M 


□ PN 


□ 


DN 


Travel to attend OTAN “hands-on” training 
at a location outside your area: 


□ VL 


□ L 


□ M 


□ PN 


□ 


DN 


Download and print out training handouts 
and materials from OTAN’s Web Site: 


□ VL 


□ L 


□ M 


□ PN 


□ 


DN 


Utilize individualized training and “how-to” 
tutorials provided on OTAN’s Web Site: 


□ VL 


□ L 


□ M 


□ PN 


□ 


DN 



0 
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BEST COPY AVAILABLE 



GENERAL USAGE 



During the program year, how often do you logon to OTAN Online? (Check one.) 

□ Five or more times per week 

□ One to four times per week 

□ One to three times month 

□ Less than once a month 

□ Sporadically— use it every day for a while, then don’t go on for weeks or months 

□ Other (Please describe in one sentence.) 



What prompted you to use the OTAN Online communication system? (Check one.) 

□ It was required as part of my agency’s 321 funding. 

□ It was offered to me as part of my work in my agency. 

□ I heard about it outside of my agency. 

□ Other (Please describe in one sentence.) 



Which of these OTAN Online areas do you use? (Check all that apply.) 

□ Mailbox □ OTAN Exchange 

□ Roundtable (electronic work groups) □ Conferences 

□ News □ Help 

If you checked MAILBOX above, please continue. Otherwise, skip to Q. 19. 

How easy do you find it to use OTAN Online’s ELECTRONIC MAIL? 

D Very Easy D Easy □ Average Q Difficult (HI Very Difficult 
What are your reasons for using OTAN ELECTRONIC MAIL? (Check all that apply.) 



□ Assist in instruction 


□ Manage program 


□ Manage staff 


□ Improve program 


□ Improve staff 


□ Meet CDE requirements 


□ Communicate with staff from 


□ Work with others on joint project 


other programs/agencies 


□ Other (Describe: 


□ Improve self 





If you checked ROUNDTABLE, please continue. Otherwise, skip to Q. 20. 



19b. When you logon to OTAN Online, how often do you use ROUNDTABLE? 

□ Everytime 

□ About 75% of the time 

□ About 50% of the time 

□ About 25% of the time 

□ Seldom 



19c. 



19d. 



20a. 



How easy do you find it to use OTAN Online’s ROUNDTABLE? 

□ Very Easy □ Easy □ Average □ Difficult □ Very Difficult 
Which ROUNDTABLE work groups have you participated in? (Check all that apply.) 



□ AD ED Leadership 


□ 


□ CASAS National Consortium 


□ 


□ CDLP Advisory 


□ 


□ CA Library Literacy 


□ 


□ Learning Net 


□ 


□ Oregon (OCABSD) 


□ 


□ SCLLN 


□ 


□ Adult Email projects 


□ 


□ ccc 


□ 


□ Consortium of Adult Education 


□ 



19e. What are your reasons for using OTAN ROUNDTABLE? (Check all that apply.) 



□ Assist in instruction 


□ 


□ Manage staff 


□ 


□ Improve staff 


□ 


□ Communicate with staff from 


□ 


other programs/agencies 


□ 


□ Improve self 





If you checked CONFERENCES/EMAIL LISTSERVs, please continue. Otherwise 
skip to Q. 21. 



20b. When you logon to OTAN Online, how often do you use CONFERENCES/EMAIL LIST 
SERVs? ' ^ . 

□ Everytime 

□ About 75% of the time 

□ About 50% of the time 

□ About 25% of the time i 

□ Seldom 
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20c. How easy do you find it to use OTAN Online’s Conferences/Email LISTSERVs? 

□ Very Easy □ Easy □ Average □ Difficult □ Very Difficult 

20d. What are your reasons for using OTAN’s Conferences/Email LISTSERVs? (Check all 
that apply.) 



□ Assist in instruction 

□ Manage staff 

□ Improve program 

□ Work with others on joint project 

□ Improve self 



□ Manage program 

□ Improve program 

□ Communicate with staff from other 
programs/agencies 

□ Other (Describe: 



20e. Please indicate how often you participate in the following LISTSERVs. 

D= Daily W=Weekly M=Monthly LM=Less than monthly 

R=Rarely N=Never 



Adult Ed Network: 


□ 


D 


□ W 


□ M 


□ LM 


□ 


R 


□ N 


Adult Literacy List: 


□ 


D 


□ W 


□ M 


□ LM 


□ 


R 


□ N 


Adult Numeracy: 


□ 


D 


□ w 


□ M 


□ LM 


□ 


R 


□ N 


Alternative Learning: 


□ 


D 


□ w 


□ M 


□ LM 


□ 


R 


□ N 


CATESOL Jobs: 


□ 


D 


□ w 


□ M 


□ LM 


□ 


R 


□ N 


DEOS-L: 


□ 


D 


□ w 


□ M 


□ LM 


□ 


R 


□ N 


DEOS News: 


□ 


D 


□ w 


□ M 


□ LM 


□ 


R 


□ N 


Edinfo: 


□ 


D 


□ w 


□ M 


□ LM 


□ 


R 


□ N 


Education Style: 


□ 


D 


□ w 


□ M 


□ LM 


□ 


R 


□ N 


Edupage: 


□ 


D 


□ w 


□ M 


□ LM 


□ 


R 


□ N 


NIFL: 


□ 


D 


□ w 


□ M 


□ LM 


□ 


R 


□ N 


National Lit. Alliance: 


□ 


D 


□ w 


□ M 


□ LM 


□ 


R 


□ N 


NWAC: 


□ 


D 


□ w 


□ M 


□ LM 


□ 


R 


□ N 


Prison Teachers: 


□ 


D 


□ w 


□ M 


□ LM 


□ 


R 


□ N 


TESL-L: 


□ 


D 


□ w 


□ M 


□ LM 


□ 


R 


□ N 


Vocational Net: 


□ 


D 


□ w 


□ M 


□ LM 


□ 


R 


□ N 



21a. In general, how satisfied are you with the OTAN Online services? (Check one.) 



□ Very Satisfied □ Satisfied □ Neutral □ Dissatisfied □ Very Dissatisfied 



21b. If you are satisfied, please describe the most important way in which the Online services 
meet your needs. 
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21c. If you are dissatisfied, please describe one way in which the Online services might better 
meet your needs. 



22. Which of these OTAN WWW areas do you use? (Check all that apply.) 

□ OTAN Forum □ Search OTAN Forum 

□ Customer Services □ OTAN Partners 

□ Local Education Agencies - □ Online Resources - 

□ NIFL Region IV Hub □ Distance Learning Project 

IF YOU DON’T USE THE OTAN FORUM, SKIP TO Q. 36. 

23. If you use the MASTER CALENDAR, please evaluate the ease of use; if not, check 
“Don’t Use.” 

□ Very Easy □ Easy □ Average □ Difficult □ Very Difficult □ Don’t Use 

24. If you use WHO’s WHO, please evaluate the ease of use; if not, check “Don’t Use.” 

□ Very Easy □ Easy □ Average □ Difficult □ Very Difficult □ Don’t Use 

25a. If you use CDE INFORMATION, please evaluate the ease of use; if not, check “Don’t 
Use.” 

□ Very Easy □ Easy □ Average □ Difficult □ Very Difficult □ Don’t Use 

25b. If you use CDE Information, what are your reasons? (Check all that apply.) 

□ Manage program □ Manage staff 

□ Improve program □ Improve staff 

□ Meet CDE requirements □ Other (Describe: 

) 

. 26a. If you use the REGIONAL RESOURCE CENTERS, please evaluate the ease of use; if 
not, check “Don’t Use.” 

□ Very Easy □ Easy □ Average □ Difficult □ Very Difficult □ Don’t Use 

26b. If you use the Regional Resource Centers, what are your reasons? (Check all that apply.) 

□ Assist in instruction □ Manage program 

□ Manage staff □ Improve program 

□ Improve staff □ Meet CDE requirements 
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□ Work with others on joint project □ Improve self 

□ Other (Describe: ) 

27a. If you use DOCUMENT LIBRARY, please evaluate the ease of use; if not, check 
“Don’t Use.” 



□ Very Easy □ Easy □ Average □ Difficult □ Very Difficult □ Don’t Use 



27b. If you use the Document Library, what are your reasons? (Check all that apply.) 



□ Assist in instruction 

□ Manage staff 

□ Improve staff 

□ Work with others on joint project 

□ Other (Describe: 



□ Manage program 

□ Improve program 

□ Meet CDE requirements 

□ Improve self 

) 



28a. If you use CURRICULA RESOURCES, please evaluate the ease of use; if not, check 
“Don’t Use.” 

□ Very Easy □ Easy □ Average □ Difficult □ Very Difficult □ Don’t Use 
28b. If you use Curricula Resources, what are your reasons? (Check all that apply.) 

□ Assist in instruction □ Improve program 

□ Improve staff □ Meet CDE requirements 

□ Work with others on joint project □ Improve self 

□ Other (Describe: ) 

29a. If you use COURSE OUTLINES, please evaluate the ease of use; if not, check “Don’t 
Use.” 



□ Very Easy □ Easy □ Average □ Difficult □ Very Difficult □ Don’t Use 
29b. If you use Course Outlines, what are your reasons? (Check all that apply.) 

□ Assist in instruction □ Improve program 

□ Improve staff □ Meet CDE requirements 

□ Work with others on joint project □ Improve self 

□ Other (Describe: ) 



30a. If you use LESSON PLANS, please evaluate the ease of use; if not, check “Don’t Use.” 
□ Very Easy □ Easy □ Average □ Difficult □ Very Difficult □ Don’t Use 
30b. If you use Lesson Plans, what are your reasons? (Check all that apply.) 
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□ Assist in instruction □ Improve program 

□ Improve staff □ Meet CDE requirements 

□ Work with others on joint project □ Improve self 

□ Other (Describe: ) 

31a. If you use REFERENCE LIBRARIES, please evaluate the ease of use; if not, check 
“Don’t Use.” 



□ Very Easy □ Easy □ Average □ Difficult □ Very Difficult □ Don’t Use 
31b. If you use Reference Libraries, what are your reasons? (Check all that apply.) 



□ Assist in instruction 

□ Manage staff 

□ Improve staff 

□ Improve self 



□ Manage program 

□ Improve program 

□ Work with others on joint project 

□ Other (Describe: 



) 



32. If you use LEGISLATIVE INFORMATION, please evaluate the ease of use; if not, 

check “Don’t Use.” 



□ Very Easy □ Easy □ Average □ Difficult □ Very Difficult □ Don’t Use 

33. If you use CODES AND REGULATIONS, please evaluate the ease of use; if not, check 
“Don’t Use.” 



□ Very Easy □ Easy □ Average □ Difficult □ Very Difficult □ Don’t Use 

34. If you use EDUCATIONAL GRANTS, please evaluate the ease of use; if not, check 
“Don’t Use.” 

□ Very Easy □ Easy □ Average □ Difficult □ Very Difficult □ Don’t Use 



35. When you access the OTAN Forum, do you use the following search features? If yes, 
check “Use” and indicate your level of satisfaction; if no, check “Don’t Use.” 



Keyword: □ Use 

□ Very Satisfied 
Specific Searches: 

□ Very Satisfied 
By-Date Search: 

□ Very Satisfied 



□ Don’t Use 

□ Satisfied □ Neutral 

□ Use □ Don’t Use 

□ Satisfied □ Neutral 
□ Use □ Don’t Use 

□ Satisfied □ Neutral 



□ Dissatisfied 


□ Very Dissatisfied 


□ Dissatisfied 


□ Very Dissatisfied 


□ Dissatisfied 


□ Very Dissatisfied 



36a. In general, how satisfied are you with OTAN WWW? 



□ Very Satisfied □ Satisfied □ Neutral □ Dissatisfied 



□ Very Dissatisfied 
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36b. 



If you are satisfied, please describe the most important way in which OTAN WWW 
services meet your needs: 



36c. If you are dissatisfied, please describe one way in which OTAN WWW services might 
better meet your needs: 



37. Please describe the two most important factors that have facilitated your use of the OTAN 
Online communication system: 



38. Please describe the two most important barriers that you have encountered in using the 
OTAN Online communication system: 



39. As a result of using OTAN, have you undertaken any activities related to your job that 
you wouldn’t have undertaken before using OTAN? 

□ Yes □ No 

If yes, please describe. 



40. Please describe any direct effects that your use of OTAN’s services have had on one or 
more of the following: 

a) program administration; 

b) instruction; 

c) program improvement; and/or 

d) staff morale. 




11 
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THANK YOU FOR COMPLETING THIS SURVEY. PLEASE MAIL YOUR 
COMPLETED SURVEY TO JUDITH ALAMPRESE IN THE ENCLOSED, 
STAMPED ENVELOPE NO LATER THAN NOVEMBER 7, 1997. 
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OTAN NON-USER SURVEY 



Name: 



Name of Agency/Organization: 

Address: 

City. State: Zip: 

1 , Which category is the closest match to your job description? (Check one.) 

□ LEA/CBO Administrator □ Clerical or Other Support Staff 

□ Classroom Teacher □ State/Federal/Other 

□ Resource Teacher/Coordinator 



In what year did you become an OTAN Online/WWW subscriber? 19 

□ Don’t Know 



Approximately how many times have you ever logged-on to OTAN? 
Approximately how many times have you visited the OTAN web site? 



times 

times 



Which of these OTAN Online areas have you ever used? (Check all that apply.) 

□ Mailbox □ OTAN Exchange 

□ Roundtable (electronic work groups) □ Conferences 

□ News □ Help 

□ None 



List the two most important reasons why you currently do not use the OTAN Online 
communication system? 



Are there any circumstances under which you would use OTAN? □ Yes 
If yes, please describe. 



What types of information would you find most useful to obtain through OTAN? 



ERIC 
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BEST COPY AVAILABLE 



OTAN User Survey 

Spring 1 999 



General Information 

la. Please review the contact information on the label and make any corrections here 



lb. If different than above, please enter the name of the person 

completing this survey: 

2. Which category is the closest match to your job description? 

I I LEA / CBO / CC Administrator Q Clerical or Other Support Staff Q State / Federal / Other 

I | Classroom Teacher Q Resource Teacher / Coordinator 



3. During Fall 1998, how often did you access OTAN’s web site? 

I | Five or more times per week Q One to four times per week 

I I One to three times per month Q Less than once a month 

I | Sporadically (use it every day for a while, then don’t go on for weeks or months) 



Use of the Redesigned Web Site 

4. Please describe your use of OTAN since the redesigned web site design was released in mid-January 1999. 
[~1 I’m using it more often than I did in Fall 1998. O I’m using it about as often as I did in Fall 1998 
I I I’m using it less often than I did in Fall 1998. Q I haven’t used it at all since it was redesigned. 

If you haven’t used the web site or use it less often, please explain why: 



5a. In the redesigned OTAN, subscribers are encouraged to create detailed User Profiles. Subscribers can chose to 
include their Profile in the OTAN member directory, which can be used by other subscribers for networking 
purposes. Also, the Profiles will be used to develop a “smart” web site that is customized to reflect individual 
users’ interests. Please check one of the following: 

I I I created a User Profile and made my information accessible to the OTAN member directory. 

I I I created a User Profile but chose NOT to include it in the OTAN member directory for the following 

reason(s): 

I I I have not created a User Profile for the following reason(s): 



erJc 



m 

nk you for your participation! Please return this survey as soon as possible in the enclosed envelope. 
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5b. Have you used the Advanced Directory search to look up User Profiles by county, department, or job category? 

□ Yes, and I plan to use it again. □ Yes, but I probably won’t use it again. 

□ No, but I plan to use it in the future Q No, and I don’t plan to use it in the future. 

If yes, how did you use this information? 



5c. Would you like to be able to search other subscribers’ User Profiles by the subject areas they are interested in'? 
□ Yes □ No 

If yes, how would you use this information? 



6. OTAN’s Menu bar on the left-hand side of the screen (with red background and white lettering) was 

redesigned. In general, how easy is it to navigate OTAN’s web site through using the redesigned Menu bar? 

□ Very Easy □ Easy □ Average □ Difficult Q Very Difficult 

If you thought the Menu bar was “Difficult” or “Very Difficult” to use, please explain why: 



7a. The “News” is now accessed from the Members home page, in the upper right-hand corner. Included in the 
News are a wide variety of alerts such as funding opportunities, survey and report notices, meetings and 
conferences, and policy input requests. Would you like News items to be emailed directly to your preferred 
address? 

□ Yes □ No 

Comments: 



7b. Would you like News items to flash on the web page as they did on OTAN Online? 
□ Yes □ No 

Comments: 



8a. Information in the OTAN Forum is constantly updated, and several sections have been extensively modified 
and/or improved with database technology. Below, please check the sections of the Forum you find useful. 

□ CDEInfo □ Master Calendar O Legislative Information □ Who’s Who 

□ Educational Grants Q For Teachers □ Document Library Q Reference Libraries 

□ CA Adult Ed History □ CA Adult Ed Products □ Codes and Regulations □ Presentations 

T 9 9 - 

you for your participation! Please return this survey as soon as possible in the enclosed envelope. 
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8b. Please suggest how any of these Forum areas from 8a could be improved to be more useful to you: 



9. The Round Table area also was redesigned. If you’ve used the redesigned Round Table area, please evaluate 
the ease of use; if not, check “Haven’t Used It.” 

("I Very Easy O Easy O Average O Difficult Q Very Difficult Q Haven’t Used It 
If you thought the Round Table area was “Difficult” or “Very Difficult” to use, or haven’t used it, explain why: 



10. To help users navigate the redesigned web site, OTAN developed an online User Guide. If you’ve used the 
new User Guide, please evaluate the ease of use; if not, check “Haven’t Used It.” 

| | Very Easy Q Easy Q Average Q Difficult Q Very Difficult Q Haven’t Used It 
If you thought the User Guide was “Difficult” or “Very Difficult” to use, or haven’t used it, explain why: 



1 1 . In general, how satisfied are you with the redesigned OTAN web site? 

| | Very Satisfied Q Satisfied O Neutral Q Dissatisfied Q Very Dissatisfied 

Comments: 



84 



ERIC you for your participation! Please return this survey as soon as possible in the enclosed envelope. 
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Applications of OTAN 



12. We are interested in learning about how you’ve used OTAN’s information and communication tools as 

resources for program improvement. Below, please give specific examples of how you’ve used OTAN to help 
accomplish any of the following activities: (a) Developing/revising curricula or identifying teaching materials; 

(b) Providing staff development or enhancing your own skills; (c) Improving program management (e.g., 
revising policies, pursuing funding opportunities, etc.); and (d) Teaching learners. (If necessary, continue on 
the reverse side or attach additional sheets.) 

(a) Develdping/revising curricula or identifying teaching materials: ~ 



(b) Providing staff development or enhancing your own skills: 



(c) Improving program management (e.g., revising policies, pursuing funding opportunities, etc.): 



(d) Teaching learners: 







RJ ink you for your participation! Please return this survey as soon as possible in the enclosed envelope. 
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Appendix B 

Graphs depicting use of the OTAN Website 
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Exhibit B-4: Number of User Sessions Recorded by the Three Most Active Providers 

June 1998 - May 1999 
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Exhibit B-5: Number of Hits and User Sessions Recorded by otan.dni.us users 

June 1998 - May 1999 
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Exhibit B-6: Number of Hits and User Sessions Recorded by kl2.ca.us users 

June 1998 - May 1999 
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communication that subscribers could undertake, such as the installation of the Email 
list server software to host online Email discussions as well as software to provide 
online multimedia content. In addition to the provision of ongoing training and 
technical assistance, OTAN’s staff conducted targeted staff development to expand 
the capacity of key system users, such as adult education agency administrators and 
Regional Resource Center managers. The expansion of the library of online materials 
continued as did the establishment of linkages with other online systems. To enable 
local agencies to be more systematic in their use of technology, OTAN offered 
technology planning and implementation workshops to adult education providers. 

• Enhancing the OTAN System: April 1997-June 1999. During this period, OTAN’s 
staff made a number of changes in the system to respond to state policy shifts as well 
as subscribers’ needs. OTAN continued to improve the communication and 
information system, provided additional training and technical assistance, further 
enhanced the information and reference services, and offered outreach and assistance 
to support technology planning. Shifts in state policy prompted OTAN to adjust the 
electronic communication system in three ways: 1) to provide free email accounts to 
agencies receiving state adult education monies; 2) to transition out of the business of 
providing both Internet access and Email, because of the increased number of OTAN 
subscribers who no longer relied on OTAN for accessing Email; and 3) to refine 
OTAN’s Web site to include all components of service. Other new developments 
during this period included the piloting of online staff development in conjunction 
with the state’s staff development provider, the development of multimedia programs, 
and the expansion of partnership with entities such as CNN to provide online adult 
learner instruction. In January 1999, OTAN’s released the redesigned Web site. In 
addition to the no-cost registration feature, it has an online User Guide to assist 
inexperienced visitors, a Java-based pull-down menu system, a new home page, a 
redesigned Round Table area, and an expanded area called “OTAN For Teachers.” 

These phases of development represented a systematic approach to the creation of an online 
communication system that was responsive to users’ needs as well as incorporated the latest 
developments in technology. During each grant period, OTAN’s staff assessed the ways in 
which the system could be improved technologically as well as substantively and made the 
adjustments accordingly. The vision of OTAN’s director and staff enabled the system to 
keep pace with new developments without becoming too advanced for its intended users. 

Technical Assistance and Training. A critical element in the development of OTAN has 
been the training and technical assistance that staff created to support the dissemination of the 
system and encourage its use. The assistance that OTAN provided not only enabled local 
adult education program staff to become familiar with the online system and its capabilities, 
but also played a critical role in building the capacity of these staff to use computers. 

OTAN’s staff leveraged their training and technical assistance expertise by working with the 
Regional Resource Centers to deliver assistance to the adult education programs in their 
regions. OTAN’s staff also collaborated with these programs in hosting training sessions that 

Conclusions and Future Directions V-2 



Abt Associates Inc. 



101 




® 

U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 

National Library of Education (NLE) 

Educational Resources Information Center (ERIC) 

REPRODUCTION RELEASE 

(Specific Document) 




I. DOCUMENT IDENTIFICATION: 



Title: C t> o-ecV^ G\ 


Peopfc.> radioes, anc| Policies-. unit bet of "f Ac. 

cvwdi "V-ec Urn'cal P)< C& taio} 


Author(s): Judith A. Alamprese, Eric Stickney 


Corporate Source: 

Abt Associates Inc. 


Publication Date: 

February 2001 



II. REPRODUCTION RELEASE: 

In order to disseminate as widely as possible timely and significant materials of interest to the educational community, documents announced in the 
monthly abstract journal of the ERIC system, Resources in Education (RIE), are usually made available to users in microfiche, reproduced paper copy, 
and electronic media, and sold through the ERIC Document Reproduction Service (EDRS). Credit is given to the source of each document and, if 
reproduction release is granted, one of the following notices is affixed to the document. 

If permission is granted to reproduce and disseminate the identified document please CHECK ONE of the following three options and sign at the bottom 
of the page. 




Level 1 



Level 2A 



Level 2B 



t 







Chet* here for Level 1 r elate , pamtftttng reproduction 
and dissemination in m to ofl ch e orotterERlC archival 
media (e.Q.. electronic) and paper copy. 



Check here for Level 2A release , permitting reproduction 
ana QiiiaiinflDOfi ro nacroocno ana in otocuonic nwii 
for ERIC archival coMection aubscriber* only 



Check here for Level 2B release, perm t tbng 
reproduction end dissemination in microiche only 



Documents vriB be p roce ss ed ss indicated provided reproduction quafity permits. 

If permi ssi on to reproduce is granted. but no boot is cheated, documents wil be processed at Level 1. 



Sign 

here,-* 

please 



/ hereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminate this document 
as indicated above. Reproduction from the ERIC microfiche or electronic media by persons other than ERIC employees and its system 
contractors requires permission from the copyright holder. Exception is made for non-profit reproduction by libraries and other service agencies 


to satisfy information needs of educators in response to discrete inquiries. 






Si9n * li " : / * — <r^=* — T) 


Printed Nama/PostocxVTid® 




^ \ 


Eric Stickney / Senior Analyst 


OrgnzatkirVAc&ms: 


Telephone: 30 l-913-0661 


FA * 301-718-3108 


Abt Associates / 4800 Montgomery Lane / Bethesda, MD 


E-Mafl Address: 

eric_stickney@abtassoc.com 


°** : 07/12/2002 







III. DOCUMENT AVAILABILITY INFORMATION (FROM NON-ERIC SOURCE): 



If permission to reproduce is not granted to ERIC, or, if you wish ERIC to cite the availability of the document from another source, 
please provide the following information regarding the availability of the document. (ERIC will not announce a document unless it is 
publicly available, and a dependable source can be specified. Contributors should also be aware that ERIC selection criteria are 
significantly more stringent for documents that cannot be made available through EDRS.) 

Publisher/Distributor: 



Address: 



Price: 



IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS HOLDER: 

If the right to grant this reproduction release is held by someone other than the addressee, please provide the appropriate name and 
address: 




V. WHERE TO SEND THIS FORM: 



Send this form to the following ERIC Clearinghouse: 



However, if solicited by the ERIC Facility, or if making an unsolicited contribution to ERIC, return this form (and the document being 
contributed) to: 



ERIC Processing and Reference Facility 
4483-A Forbes Boulevard 
Lanham, Maryland 20706 



Telephone: 
Toll Free: 
FAX: 
e-mail: 
WWW: 



301-552-4200 

800-799-3742 

301-552-4700 

ericfac@inet.ed.gov 

http://ericfacility.org 



EFF-088 (Rev. 2/2001) 




